2024-03-16T09:57:19.243 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-16T09:57:19.243 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} --archive /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360 --owner scheduled_pdonnell@teuthology --verbose --name pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360/orig.config.yaml 2024-03-16T09:57:19.284 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T09:57:19.359 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360 branch: wip-batrick-testing-20240314.183243 description: fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} email: pdonnell@redhat.com first_in_suite: false job_id: '7602360' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-batrick-testing-20240314.183243 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240314.183243 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 seed: 4770 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af sleep_before_teardown: 0 subset: 498/512 suite: fs suite_branch: wip-batrick-testing-20240314.183243 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi017.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAzG3WuJoO9CsYpabwEtXZIEsNWtuwVD9c72O5USbYQ8CXtNNbr/289XNGIWmEdcOxZjFtJOiM+NbxM2k1OpnIk= smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKvDj+uaglxBy4cNsSC8W66C4LA0RrviET8P2Pr+TEqKog/O1npB2zywdCmM4QOM8TIu5s5VxLpqZoxG/2ZLGzg= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_cephfs_shell teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-15_13:00:41 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-16T09:57:19.359 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa; will attempt to use it 2024-03-16T09:57:19.360 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks 2024-03-16T09:57:19.360 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-16T09:57:19.361 INFO:teuthology.task.internal:Checking packages... 2024-03-16T09:57:19.385 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af' 2024-03-16T09:57:19.385 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T09:57:19.385 INFO:teuthology.packaging:ref: None 2024-03-16T09:57:19.385 INFO:teuthology.packaging:tag: None 2024-03-16T09:57:19.385 INFO:teuthology.packaging:branch: wip-batrick-testing-20240314.183243 2024-03-16T09:57:19.385 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T09:57:19.385 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-batrick-testing-20240314.183243 2024-03-16T09:57:19.637 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2162-g3d78f5ba-1jammy 2024-03-16T09:57:19.638 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-16T09:57:19.645 INFO:teuthology.task.internal:no buildpackages task found 2024-03-16T09:57:19.645 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-16T09:57:19.652 INFO:teuthology.task.internal:Saving configuration 2024-03-16T09:57:19.666 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-16T09:57:19.671 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T09:57:19.696 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 09:51:15.341845', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAEH96mtqgpj6caf4ETwBCoO+S74ZUREEccuZZl++fXB'} 2024-03-16T09:57:19.716 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 09:51:15.342971', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILnN/lCnDsWML2x/iRkP4nrpr2SjffowzMB2tUGwmnAO'} 2024-03-16T09:57:19.716 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-16T09:57:19.722 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-03-16T09:57:19.722 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-03-16T09:57:19.722 INFO:teuthology.run_tasks:Running task console_log... 2024-03-16T09:57:19.784 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fcbf60ec3a0>, signals=[15]) 2024-03-16T09:57:19.785 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-16T09:57:19.791 INFO:teuthology.task.internal:Opening connections... 2024-03-16T09:57:19.791 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi017.front.sepia.ceph.com 2024-03-16T09:57:19.793 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:57:19.873 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2024-03-16T09:57:19.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:57:19.949 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-16T09:57:19.955 DEBUG:teuthology.orchestra.run.smithi017:> uname -m 2024-03-16T09:57:19.961 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64 2024-03-16T09:57:19.961 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release 2024-03-16T09:57:20.007 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-16T09:57:20.007 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="Ubuntu" 2024-03-16T09:57:20.007 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="22.04" 2024-03-16T09:57:20.008 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-16T09:57:20.008 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_CODENAME=jammy 2024-03-16T09:57:20.008 INFO:teuthology.orchestra.run.smithi017.stdout:ID=ubuntu 2024-03-16T09:57:20.008 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE=debian 2024-03-16T09:57:20.008 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-16T09:57:20.008 INFO:teuthology.orchestra.run.smithi017.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-16T09:57:20.008 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-16T09:57:20.008 INFO:teuthology.orchestra.run.smithi017.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-16T09:57:20.008 INFO:teuthology.orchestra.run.smithi017.stdout:UBUNTU_CODENAME=jammy 2024-03-16T09:57:20.009 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2024-03-16T09:57:20.036 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-03-16T09:57:20.040 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-03-16T09:57:20.040 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-03-16T09:57:20.087 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-16T09:57:20.087 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="Ubuntu" 2024-03-16T09:57:20.087 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="22.04" 2024-03-16T09:57:20.087 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-16T09:57:20.087 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_CODENAME=jammy 2024-03-16T09:57:20.088 INFO:teuthology.orchestra.run.smithi174.stdout:ID=ubuntu 2024-03-16T09:57:20.088 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE=debian 2024-03-16T09:57:20.088 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-16T09:57:20.088 INFO:teuthology.orchestra.run.smithi174.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-16T09:57:20.088 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-16T09:57:20.088 INFO:teuthology.orchestra.run.smithi174.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-16T09:57:20.088 INFO:teuthology.orchestra.run.smithi174.stdout:UBUNTU_CODENAME=jammy 2024-03-16T09:57:20.088 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-03-16T09:57:20.110 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-16T09:57:20.119 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-16T09:57:20.125 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-16T09:57:20.125 DEBUG:teuthology.orchestra.run.smithi017:> test '!' -e /home/ubuntu/cephtest 2024-03-16T09:57:20.127 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2024-03-16T09:57:20.135 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-16T09:57:20.142 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-16T09:57:20.142 DEBUG:teuthology.orchestra.run.smithi017:> test -z $(ls -A /var/lib/ceph) 2024-03-16T09:57:20.173 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2024-03-16T09:57:20.194 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-16T09:57:20.251 INFO:teuthology.run_tasks:Running task kernel... 2024-03-16T09:57:20.264 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-16T09:57:20.265 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-16T09:57:20.265 DEBUG:teuthology.orchestra.run.smithi017:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T09:57:20.265 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T09:57:20.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:57:20.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T09:57:20.270 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2024-03-16T09:57:20.271 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2024-03-16T09:57:20.316 INFO:teuthology.orchestra.run.smithi017.stdout:5.15.0-56-generic 2024-03-16T09:57:20.317 INFO:teuthology.orchestra.run.smithi174.stdout:5.15.0-56-generic 2024-03-16T09:57:20.317 INFO:teuthology.task.kernel:Running kernel on smithi017: 5.15.0-56-generic 2024-03-16T09:57:20.317 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-get clean 2024-03-16T09:57:20.317 INFO:teuthology.task.kernel:Running kernel on smithi174: 5.15.0-56-generic 2024-03-16T09:57:20.317 DEBUG:teuthology.orchestra.run.smithi174:> sudo apt-get clean 2024-03-16T09:57:20.438 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-get update 2024-03-16T09:57:20.447 DEBUG:teuthology.orchestra.run.smithi174:> sudo apt-get update 2024-03-16T09:57:20.601 INFO:teuthology.orchestra.run.smithi174.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-16T09:57:20.604 INFO:teuthology.orchestra.run.smithi017.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-16T09:57:20.622 INFO:teuthology.orchestra.run.smithi174.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-16T09:57:20.705 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-16T09:57:20.706 INFO:teuthology.orchestra.run.smithi174.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-16T09:57:20.713 INFO:teuthology.orchestra.run.smithi174.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-16T09:57:20.789 INFO:teuthology.orchestra.run.smithi017.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-16T09:57:20.842 INFO:teuthology.orchestra.run.smithi017.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,246 kB] 2024-03-16T09:57:20.859 INFO:teuthology.orchestra.run.smithi174.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [587 kB] 2024-03-16T09:57:20.969 INFO:teuthology.orchestra.run.smithi174.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,462 kB] 2024-03-16T09:57:21.037 INFO:teuthology.orchestra.run.smithi174.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [285 kB] 2024-03-16T09:57:21.046 INFO:teuthology.orchestra.run.smithi174.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-16T09:57:21.047 INFO:teuthology.orchestra.run.smithi174.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-16T09:57:21.048 INFO:teuthology.orchestra.run.smithi174.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,559 kB] 2024-03-16T09:57:21.080 INFO:teuthology.orchestra.run.smithi174.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [259 kB] 2024-03-16T09:57:21.084 INFO:teuthology.orchestra.run.smithi174.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-16T09:57:21.084 INFO:teuthology.orchestra.run.smithi174.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,057 kB] 2024-03-16T09:57:21.100 INFO:teuthology.orchestra.run.smithi174.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [695 kB] 2024-03-16T09:57:21.108 INFO:teuthology.orchestra.run.smithi174.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [239 kB] 2024-03-16T09:57:21.108 INFO:teuthology.orchestra.run.smithi017.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [421 kB] 2024-03-16T09:57:21.111 INFO:teuthology.orchestra.run.smithi174.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-16T09:57:21.112 INFO:teuthology.orchestra.run.smithi174.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-16T09:57:21.115 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [225 kB] 2024-03-16T09:57:21.119 INFO:teuthology.orchestra.run.smithi017.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-16T09:57:21.119 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,531 kB] 2024-03-16T09:57:21.146 INFO:teuthology.orchestra.run.smithi174.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-16T09:57:21.147 INFO:teuthology.orchestra.run.smithi174.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [421 kB] 2024-03-16T09:57:21.147 INFO:teuthology.orchestra.run.smithi174.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-16T09:57:21.148 INFO:teuthology.orchestra.run.smithi017.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-16T09:57:21.148 INFO:teuthology.orchestra.run.smithi174.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-16T09:57:21.148 INFO:teuthology.orchestra.run.smithi174.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-16T09:57:21.148 INFO:teuthology.orchestra.run.smithi174.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-16T09:57:21.149 INFO:teuthology.orchestra.run.smithi174.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-16T09:57:21.150 INFO:teuthology.orchestra.run.smithi174.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-16T09:57:21.151 INFO:teuthology.orchestra.run.smithi017.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-16T09:57:21.156 INFO:teuthology.orchestra.run.smithi017.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [255 kB] 2024-03-16T09:57:21.156 INFO:teuthology.orchestra.run.smithi017.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-16T09:57:21.156 INFO:teuthology.orchestra.run.smithi017.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [598 kB] 2024-03-16T09:57:21.166 INFO:teuthology.orchestra.run.smithi174.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-16T09:57:21.167 INFO:teuthology.orchestra.run.smithi017.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [850 kB] 2024-03-16T09:57:21.175 INFO:teuthology.orchestra.run.smithi017.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-03-16T09:57:21.177 INFO:teuthology.orchestra.run.smithi017.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-16T09:57:21.177 INFO:teuthology.orchestra.run.smithi017.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-16T09:57:21.178 INFO:teuthology.orchestra.run.smithi017.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-16T09:57:21.188 INFO:teuthology.orchestra.run.smithi174.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-16T09:57:21.188 INFO:teuthology.orchestra.run.smithi174.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.6 kB] 2024-03-16T09:57:21.188 INFO:teuthology.orchestra.run.smithi174.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-16T09:57:21.192 INFO:teuthology.orchestra.run.smithi017.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-16T09:57:21.192 INFO:teuthology.orchestra.run.smithi017.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-16T09:57:21.290 INFO:teuthology.orchestra.run.smithi017.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,462 kB] 2024-03-16T09:57:21.416 INFO:teuthology.orchestra.run.smithi174.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,246 kB] 2024-03-16T09:57:21.580 INFO:teuthology.orchestra.run.smithi174.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [225 kB] 2024-03-16T09:57:21.587 INFO:teuthology.orchestra.run.smithi017.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [587 kB] 2024-03-16T09:57:21.592 INFO:teuthology.orchestra.run.smithi174.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-16T09:57:21.593 INFO:teuthology.orchestra.run.smithi174.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,531 kB] 2024-03-16T09:57:21.626 INFO:teuthology.orchestra.run.smithi017.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [285 kB] 2024-03-16T09:57:21.641 INFO:teuthology.orchestra.run.smithi017.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-16T09:57:21.642 INFO:teuthology.orchestra.run.smithi017.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-16T09:57:21.644 INFO:teuthology.orchestra.run.smithi017.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,559 kB] 2024-03-16T09:57:21.673 INFO:teuthology.orchestra.run.smithi174.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-16T09:57:21.673 INFO:teuthology.orchestra.run.smithi174.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [255 kB] 2024-03-16T09:57:21.682 INFO:teuthology.orchestra.run.smithi174.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-16T09:57:21.682 INFO:teuthology.orchestra.run.smithi174.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [850 kB] 2024-03-16T09:57:21.702 INFO:teuthology.orchestra.run.smithi017.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [259 kB] 2024-03-16T09:57:21.706 INFO:teuthology.orchestra.run.smithi174.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [598 kB] 2024-03-16T09:57:21.708 INFO:teuthology.orchestra.run.smithi017.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-16T09:57:21.884 INFO:teuthology.orchestra.run.smithi174.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-03-16T09:57:21.928 INFO:teuthology.orchestra.run.smithi017.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,057 kB] 2024-03-16T09:57:22.282 INFO:teuthology.orchestra.run.smithi174.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-16T09:57:22.289 INFO:teuthology.orchestra.run.smithi174.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-16T09:57:22.302 INFO:teuthology.orchestra.run.smithi174.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-16T09:57:22.303 INFO:teuthology.orchestra.run.smithi174.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-16T09:57:22.305 INFO:teuthology.orchestra.run.smithi174.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-16T09:57:22.566 INFO:teuthology.orchestra.run.smithi017.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [695 kB] 2024-03-16T09:57:22.604 INFO:teuthology.orchestra.run.smithi017.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [239 kB] 2024-03-16T09:57:22.616 INFO:teuthology.orchestra.run.smithi017.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-16T09:57:22.617 INFO:teuthology.orchestra.run.smithi017.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-16T09:57:22.618 INFO:teuthology.orchestra.run.smithi017.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-16T09:57:22.620 INFO:teuthology.orchestra.run.smithi017.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-16T09:57:22.621 INFO:teuthology.orchestra.run.smithi017.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-16T09:57:22.621 INFO:teuthology.orchestra.run.smithi017.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-16T09:57:22.647 INFO:teuthology.orchestra.run.smithi017.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-16T09:57:22.651 INFO:teuthology.orchestra.run.smithi017.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-16T09:57:22.815 INFO:teuthology.orchestra.run.smithi017.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-16T09:57:22.898 INFO:teuthology.orchestra.run.smithi017.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-16T09:57:22.982 INFO:teuthology.orchestra.run.smithi017.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-16T09:57:23.063 INFO:teuthology.orchestra.run.smithi017.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.6 kB] 2024-03-16T09:57:23.103 INFO:teuthology.orchestra.run.smithi017.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-16T09:57:26.580 INFO:teuthology.orchestra.run.smithi174.stdout:Fetched 12.2 MB in 2s (6,756 kB/s) 2024-03-16T09:57:27.713 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 12.2 MB in 3s (4,694 kB/s) 2024-03-16T09:57:27.929 INFO:teuthology.orchestra.run.smithi174.stdout:Reading package lists... 2024-03-16T09:57:27.946 DEBUG:teuthology.orchestra.run.smithi174:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-16T09:57:27.996 INFO:teuthology.orchestra.run.smithi174.stdout:Reading package lists... 2024-03-16T09:57:28.168 INFO:teuthology.orchestra.run.smithi174.stdout:Building dependency tree... 2024-03-16T09:57:28.169 INFO:teuthology.orchestra.run.smithi174.stdout:Reading state information... 2024-03-16T09:57:28.318 INFO:teuthology.orchestra.run.smithi174.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T09:57:28.318 INFO:teuthology.orchestra.run.smithi174.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T09:57:28.318 INFO:teuthology.orchestra.run.smithi174.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T09:57:28.318 INFO:teuthology.orchestra.run.smithi174.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T09:57:28.318 INFO:teuthology.orchestra.run.smithi174.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-16T09:57:28.319 INFO:teuthology.orchestra.run.smithi174.stdout: python2.7-minimal 2024-03-16T09:57:28.319 INFO:teuthology.orchestra.run.smithi174.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T09:57:28.319 INFO:teuthology.orchestra.run.smithi174.stdout:The following additional packages will be installed: 2024-03-16T09:57:28.319 INFO:teuthology.orchestra.run.smithi174.stdout: linux-generic linux-headers-5.15.0-100 linux-headers-5.15.0-100-generic 2024-03-16T09:57:28.319 INFO:teuthology.orchestra.run.smithi174.stdout: linux-headers-generic linux-image-5.15.0-100-generic 2024-03-16T09:57:28.320 INFO:teuthology.orchestra.run.smithi174.stdout: linux-modules-5.15.0-100-generic linux-modules-extra-5.15.0-100-generic 2024-03-16T09:57:28.321 INFO:teuthology.orchestra.run.smithi174.stdout:Suggested packages: 2024-03-16T09:57:28.321 INFO:teuthology.orchestra.run.smithi174.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-16T09:57:28.321 INFO:teuthology.orchestra.run.smithi174.stdout:Recommended packages: 2024-03-16T09:57:28.321 INFO:teuthology.orchestra.run.smithi174.stdout: thermald 2024-03-16T09:57:28.370 INFO:teuthology.orchestra.run.smithi174.stdout:The following NEW packages will be installed: 2024-03-16T09:57:28.370 INFO:teuthology.orchestra.run.smithi174.stdout: linux-headers-5.15.0-100 linux-headers-5.15.0-100-generic 2024-03-16T09:57:28.370 INFO:teuthology.orchestra.run.smithi174.stdout: linux-image-5.15.0-100-generic linux-modules-5.15.0-100-generic 2024-03-16T09:57:28.370 INFO:teuthology.orchestra.run.smithi174.stdout: linux-modules-extra-5.15.0-100-generic 2024-03-16T09:57:28.371 INFO:teuthology.orchestra.run.smithi174.stdout:The following packages will be upgraded: 2024-03-16T09:57:28.372 INFO:teuthology.orchestra.run.smithi174.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-16T09:57:28.466 INFO:teuthology.orchestra.run.smithi174.stdout:3 upgraded, 5 newly installed, 0 to remove and 297 not upgraded. 2024-03-16T09:57:28.466 INFO:teuthology.orchestra.run.smithi174.stdout:Need to get 113 MB of archives. 2024-03-16T09:57:28.466 INFO:teuthology.orchestra.run.smithi174.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-16T09:57:28.466 INFO:teuthology.orchestra.run.smithi174.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-100-generic amd64 5.15.0-100.110 [22.6 MB] 2024-03-16T09:57:28.984 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-03-16T09:57:29.002 DEBUG:teuthology.orchestra.run.smithi017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-16T09:57:29.053 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-03-16T09:57:29.235 INFO:teuthology.orchestra.run.smithi017.stdout:Building dependency tree... 2024-03-16T09:57:29.236 INFO:teuthology.orchestra.run.smithi017.stdout:Reading state information... 2024-03-16T09:57:29.393 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T09:57:29.394 INFO:teuthology.orchestra.run.smithi017.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T09:57:29.394 INFO:teuthology.orchestra.run.smithi017.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T09:57:29.394 INFO:teuthology.orchestra.run.smithi017.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T09:57:29.394 INFO:teuthology.orchestra.run.smithi017.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-16T09:57:29.394 INFO:teuthology.orchestra.run.smithi017.stdout: python2.7-minimal 2024-03-16T09:57:29.394 INFO:teuthology.orchestra.run.smithi017.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T09:57:29.395 INFO:teuthology.orchestra.run.smithi017.stdout:The following additional packages will be installed: 2024-03-16T09:57:29.395 INFO:teuthology.orchestra.run.smithi017.stdout: linux-generic linux-headers-5.15.0-100 linux-headers-5.15.0-100-generic 2024-03-16T09:57:29.395 INFO:teuthology.orchestra.run.smithi017.stdout: linux-headers-generic linux-image-5.15.0-100-generic 2024-03-16T09:57:29.396 INFO:teuthology.orchestra.run.smithi017.stdout: linux-modules-5.15.0-100-generic linux-modules-extra-5.15.0-100-generic 2024-03-16T09:57:29.397 INFO:teuthology.orchestra.run.smithi017.stdout:Suggested packages: 2024-03-16T09:57:29.397 INFO:teuthology.orchestra.run.smithi017.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-16T09:57:29.397 INFO:teuthology.orchestra.run.smithi017.stdout:Recommended packages: 2024-03-16T09:57:29.397 INFO:teuthology.orchestra.run.smithi017.stdout: thermald 2024-03-16T09:57:29.446 INFO:teuthology.orchestra.run.smithi017.stdout:The following NEW packages will be installed: 2024-03-16T09:57:29.446 INFO:teuthology.orchestra.run.smithi017.stdout: linux-headers-5.15.0-100 linux-headers-5.15.0-100-generic 2024-03-16T09:57:29.446 INFO:teuthology.orchestra.run.smithi017.stdout: linux-image-5.15.0-100-generic linux-modules-5.15.0-100-generic 2024-03-16T09:57:29.447 INFO:teuthology.orchestra.run.smithi017.stdout: linux-modules-extra-5.15.0-100-generic 2024-03-16T09:57:29.448 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages will be upgraded: 2024-03-16T09:57:29.448 INFO:teuthology.orchestra.run.smithi017.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-16T09:57:29.507 INFO:teuthology.orchestra.run.smithi017.stdout:3 upgraded, 5 newly installed, 0 to remove and 297 not upgraded. 2024-03-16T09:57:29.507 INFO:teuthology.orchestra.run.smithi017.stdout:Need to get 113 MB of archives. 2024-03-16T09:57:29.507 INFO:teuthology.orchestra.run.smithi017.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-16T09:57:29.507 INFO:teuthology.orchestra.run.smithi017.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-100-generic amd64 5.15.0-100.110 [22.6 MB] 2024-03-16T09:57:29.526 INFO:teuthology.orchestra.run.smithi174.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-100-generic amd64 5.15.0-100.110 [11.5 MB] 2024-03-16T09:57:29.793 INFO:teuthology.orchestra.run.smithi174.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-100-generic amd64 5.15.0-100.110 [63.8 MB] 2024-03-16T09:57:31.005 INFO:teuthology.orchestra.run.smithi017.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-100-generic amd64 5.15.0-100.110 [11.5 MB] 2024-03-16T09:57:31.386 INFO:teuthology.orchestra.run.smithi017.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-100-generic amd64 5.15.0-100.110 [63.8 MB] 2024-03-16T09:57:31.608 INFO:teuthology.orchestra.run.smithi174.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.100.97 [1,696 B] 2024-03-16T09:57:31.608 INFO:teuthology.orchestra.run.smithi174.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.100.97 [2,454 B] 2024-03-16T09:57:31.608 INFO:teuthology.orchestra.run.smithi174.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-100 all 5.15.0-100.110 [12.3 MB] 2024-03-16T09:57:31.864 INFO:teuthology.orchestra.run.smithi174.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-100-generic amd64 5.15.0-100.110 [2,858 kB] 2024-03-16T09:57:31.924 INFO:teuthology.orchestra.run.smithi174.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.100.97 [2,292 B] 2024-03-16T09:57:32.315 INFO:teuthology.orchestra.run.smithi174.stdout:Fetched 113 MB in 4s (31.9 MB/s) 2024-03-16T09:57:32.486 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package linux-modules-5.15.0-100-generic. 2024-03-16T09:57:32.753 INFO:teuthology.orchestra.run.smithi017.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.100.97 [1,696 B] 2024-03-16T09:57:32.856 INFO:teuthology.orchestra.run.smithi017.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.100.97 [2,454 B] 2024-03-16T09:57:32.856 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-100 all 5.15.0-100.110 [12.3 MB] 2024-03-16T09:57:33.009 INFO:teuthology.orchestra.run.smithi017.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-100-generic amd64 5.15.0-100.110 [2,858 kB] 2024-03-16T09:57:33.032 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.100.97 [2,292 B] 2024-03-16T09:57:33.498 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 113 MB in 4s (31.1 MB/s) 2024-03-16T09:57:33.685 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package linux-modules-5.15.0-100-generic. 2024-03-16T09:57:34.447 INFO:teuthology.orchestra.run.smithi174.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-16T09:57:34.451 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../0-linux-modules-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T09:57:34.765 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking linux-modules-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:35.663 INFO:teuthology.orchestra.run.smithi017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-16T09:57:35.668 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../0-linux-modules-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T09:57:35.915 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-modules-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:36.807 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package linux-image-5.15.0-100-generic. 2024-03-16T09:57:36.829 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../1-linux-image-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T09:57:36.938 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:37.452 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-100-generic. 2024-03-16T09:57:37.472 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T09:57:37.510 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking linux-modules-extra-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:38.140 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package linux-image-5.15.0-100-generic. 2024-03-16T09:57:38.162 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../1-linux-image-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T09:57:38.237 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:38.751 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-100-generic. 2024-03-16T09:57:38.773 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T09:57:38.809 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-modules-extra-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:44.057 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../3-linux-generic_5.15.0.100.97_amd64.deb ... 2024-03-16T09:57:44.182 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking linux-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-16T09:57:44.506 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.100.97_amd64.deb ... 2024-03-16T09:57:44.644 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking linux-image-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-16T09:57:44.930 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package linux-headers-5.15.0-100. 2024-03-16T09:57:44.952 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../5-linux-headers-5.15.0-100_5.15.0-100.110_all.deb ... 2024-03-16T09:57:44.996 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking linux-headers-5.15.0-100 (5.15.0-100.110) ... 2024-03-16T09:57:45.265 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../3-linux-generic_5.15.0.100.97_amd64.deb ... 2024-03-16T09:57:45.392 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-16T09:57:45.758 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.100.97_amd64.deb ... 2024-03-16T09:57:45.887 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-image-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-16T09:57:46.181 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package linux-headers-5.15.0-100. 2024-03-16T09:57:46.192 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../5-linux-headers-5.15.0-100_5.15.0-100.110_all.deb ... 2024-03-16T09:57:46.231 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-headers-5.15.0-100 (5.15.0-100.110) ... 2024-03-16T09:57:51.291 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package linux-headers-5.15.0-100-generic. 2024-03-16T09:57:51.328 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../6-linux-headers-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T09:57:51.366 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking linux-headers-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:51.553 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package linux-headers-5.15.0-100-generic. 2024-03-16T09:57:51.590 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../6-linux-headers-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T09:57:51.627 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-headers-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:54.163 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.100.97_amd64.deb ... 2024-03-16T09:57:54.299 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.100.97_amd64.deb ... 2024-03-16T09:57:54.300 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking linux-headers-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-16T09:57:54.423 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-headers-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-16T09:57:54.657 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up linux-headers-5.15.0-100 (5.15.0-100.110) ... 2024-03-16T09:57:54.757 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-headers-5.15.0-100 (5.15.0-100.110) ... 2024-03-16T09:57:54.779 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up linux-headers-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:54.861 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-headers-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:54.897 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up linux-headers-generic (5.15.0.100.97) ... 2024-03-16T09:57:54.979 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-headers-generic (5.15.0.100.97) ... 2024-03-16T09:57:55.000 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:55.083 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:56.398 INFO:teuthology.orchestra.run.smithi174.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-16T09:57:56.398 INFO:teuthology.orchestra.run.smithi174.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-16T09:57:56.398 INFO:teuthology.orchestra.run.smithi174.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-100-generic 2024-03-16T09:57:56.398 INFO:teuthology.orchestra.run.smithi174.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-100-generic 2024-03-16T09:57:56.525 INFO:teuthology.orchestra.run.smithi017.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-16T09:57:56.525 INFO:teuthology.orchestra.run.smithi017.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-16T09:57:56.525 INFO:teuthology.orchestra.run.smithi017.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-100-generic 2024-03-16T09:57:56.525 INFO:teuthology.orchestra.run.smithi017.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-100-generic 2024-03-16T09:57:56.876 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-modules-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:56.901 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up linux-modules-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:58.303 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-modules-extra-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:58.353 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up linux-modules-extra-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:57:59.738 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-image-generic (5.15.0.100.97) ... 2024-03-16T09:57:59.779 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up linux-image-generic (5.15.0.100.97) ... 2024-03-16T09:57:59.863 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-generic (5.15.0.100.97) ... 2024-03-16T09:57:59.905 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up linux-generic (5.15.0.100.97) ... 2024-03-16T09:58:00.014 INFO:teuthology.orchestra.run.smithi174.stdout:Processing triggers for linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:58:00.015 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T09:58:00.079 INFO:teuthology.orchestra.run.smithi174.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-16T09:58:00.079 INFO:teuthology.orchestra.run.smithi174.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-100-generic 2024-03-16T09:58:00.081 INFO:teuthology.orchestra.run.smithi017.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-16T09:58:00.081 INFO:teuthology.orchestra.run.smithi017.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-100-generic 2024-03-16T09:58:14.015 INFO:teuthology.orchestra.run.smithi174.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-16T09:58:14.015 INFO:teuthology.orchestra.run.smithi174.stdout:Sourcing file `/etc/default/grub' 2024-03-16T09:58:14.039 INFO:teuthology.orchestra.run.smithi174.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-16T09:58:14.059 INFO:teuthology.orchestra.run.smithi174.stdout:Generating grub configuration file ... 2024-03-16T09:58:14.302 INFO:teuthology.orchestra.run.smithi174.stdout:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-16T09:58:14.318 INFO:teuthology.orchestra.run.smithi174.stdout:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-16T09:58:14.411 INFO:teuthology.orchestra.run.smithi017.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-16T09:58:14.411 INFO:teuthology.orchestra.run.smithi017.stdout:Sourcing file `/etc/default/grub' 2024-03-16T09:58:14.432 INFO:teuthology.orchestra.run.smithi017.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-16T09:58:14.447 INFO:teuthology.orchestra.run.smithi017.stdout:Generating grub configuration file ... 2024-03-16T09:58:14.668 INFO:teuthology.orchestra.run.smithi174.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-16T09:58:14.673 INFO:teuthology.orchestra.run.smithi174.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-16T09:58:14.697 INFO:teuthology.orchestra.run.smithi017.stdout:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-16T09:58:14.710 INFO:teuthology.orchestra.run.smithi017.stdout:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-16T09:58:14.744 INFO:teuthology.orchestra.run.smithi174.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-16T09:58:14.749 INFO:teuthology.orchestra.run.smithi174.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-16T09:58:14.912 INFO:teuthology.orchestra.run.smithi174.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-16T09:58:14.912 INFO:teuthology.orchestra.run.smithi174.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-16T09:58:14.912 INFO:teuthology.orchestra.run.smithi174.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-16T09:58:14.954 INFO:teuthology.orchestra.run.smithi174.stdout:done 2024-03-16T09:58:15.011 INFO:teuthology.orchestra.run.smithi017.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-16T09:58:15.017 INFO:teuthology.orchestra.run.smithi017.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-16T09:58:15.089 INFO:teuthology.orchestra.run.smithi017.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-16T09:58:15.095 INFO:teuthology.orchestra.run.smithi017.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-16T09:58:15.253 INFO:teuthology.orchestra.run.smithi017.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-16T09:58:15.253 INFO:teuthology.orchestra.run.smithi017.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-16T09:58:15.253 INFO:teuthology.orchestra.run.smithi017.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-16T09:58:15.284 INFO:teuthology.orchestra.run.smithi017.stdout:done 2024-03-16T09:58:15.420 DEBUG:teuthology.orchestra.run.smithi174:> dpkg -s linux-image-generic 2024-03-16T09:58:15.454 INFO:teuthology.orchestra.run.smithi174.stdout:Package: linux-image-generic 2024-03-16T09:58:15.455 INFO:teuthology.orchestra.run.smithi174.stdout:Status: install ok installed 2024-03-16T09:58:15.455 INFO:teuthology.orchestra.run.smithi174.stdout:Priority: optional 2024-03-16T09:58:15.455 INFO:teuthology.orchestra.run.smithi174.stdout:Section: kernel 2024-03-16T09:58:15.455 INFO:teuthology.orchestra.run.smithi174.stdout:Installed-Size: 21 2024-03-16T09:58:15.455 INFO:teuthology.orchestra.run.smithi174.stdout:Maintainer: Ubuntu Kernel Team 2024-03-16T09:58:15.455 INFO:teuthology.orchestra.run.smithi174.stdout:Architecture: amd64 2024-03-16T09:58:15.455 INFO:teuthology.orchestra.run.smithi174.stdout:Source: linux-meta 2024-03-16T09:58:15.455 INFO:teuthology.orchestra.run.smithi174.stdout:Version: 5.15.0.100.97 2024-03-16T09:58:15.455 INFO:teuthology.orchestra.run.smithi174.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-16T09:58:15.455 INFO:teuthology.orchestra.run.smithi174.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-16T09:58:15.455 INFO:teuthology.orchestra.run.smithi174.stdout:Recommends: thermald 2024-03-16T09:58:15.455 INFO:teuthology.orchestra.run.smithi174.stdout:Description: Generic Linux kernel image 2024-03-16T09:58:15.456 INFO:teuthology.orchestra.run.smithi174.stdout: This package will always depend on the latest generic kernel image 2024-03-16T09:58:15.456 INFO:teuthology.orchestra.run.smithi174.stdout: available. 2024-03-16T09:58:15.456 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-100-generic 2024-03-16T09:58:15.456 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-16T09:58:15.456 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-16T09:58:15.456 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-16T09:58:15.456 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi174.front.sepia.ceph.com, path=None, version=distro) 2024-03-16T09:58:15.456 DEBUG:teuthology.orchestra.run.smithi174:> sudo apt-get clean 2024-03-16T09:58:15.620 DEBUG:teuthology.orchestra.run.smithi174:> sudo apt-get update 2024-03-16T09:58:15.640 DEBUG:teuthology.orchestra.run.smithi017:> dpkg -s linux-image-generic 2024-03-16T09:58:15.657 INFO:teuthology.orchestra.run.smithi017.stdout:Package: linux-image-generic 2024-03-16T09:58:15.657 INFO:teuthology.orchestra.run.smithi017.stdout:Status: install ok installed 2024-03-16T09:58:15.658 INFO:teuthology.orchestra.run.smithi017.stdout:Priority: optional 2024-03-16T09:58:15.658 INFO:teuthology.orchestra.run.smithi017.stdout:Section: kernel 2024-03-16T09:58:15.658 INFO:teuthology.orchestra.run.smithi017.stdout:Installed-Size: 21 2024-03-16T09:58:15.658 INFO:teuthology.orchestra.run.smithi017.stdout:Maintainer: Ubuntu Kernel Team 2024-03-16T09:58:15.658 INFO:teuthology.orchestra.run.smithi017.stdout:Architecture: amd64 2024-03-16T09:58:15.658 INFO:teuthology.orchestra.run.smithi017.stdout:Source: linux-meta 2024-03-16T09:58:15.658 INFO:teuthology.orchestra.run.smithi017.stdout:Version: 5.15.0.100.97 2024-03-16T09:58:15.658 INFO:teuthology.orchestra.run.smithi017.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-16T09:58:15.658 INFO:teuthology.orchestra.run.smithi017.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-16T09:58:15.658 INFO:teuthology.orchestra.run.smithi017.stdout:Recommends: thermald 2024-03-16T09:58:15.658 INFO:teuthology.orchestra.run.smithi017.stdout:Description: Generic Linux kernel image 2024-03-16T09:58:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: This package will always depend on the latest generic kernel image 2024-03-16T09:58:15.658 INFO:teuthology.orchestra.run.smithi017.stdout: available. 2024-03-16T09:58:15.659 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-100-generic 2024-03-16T09:58:15.659 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-16T09:58:15.659 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-16T09:58:15.659 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-16T09:58:15.659 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi017.front.sepia.ceph.com, path=None, version=distro) 2024-03-16T09:58:15.659 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-get clean 2024-03-16T09:58:15.832 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-get update 2024-03-16T09:58:15.875 INFO:teuthology.orchestra.run.smithi174.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-16T09:58:15.875 INFO:teuthology.orchestra.run.smithi174.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-16T09:58:15.946 INFO:teuthology.orchestra.run.smithi174.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-16T09:58:15.949 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-16T09:58:15.961 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-16T09:58:15.981 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-16T09:58:16.028 INFO:teuthology.orchestra.run.smithi174.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-16T09:58:16.069 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-16T09:58:17.218 INFO:teuthology.orchestra.run.smithi174.stdout:Reading package lists... 2024-03-16T09:58:17.235 DEBUG:teuthology.orchestra.run.smithi174:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-16T09:58:17.298 INFO:teuthology.orchestra.run.smithi174.stdout:Reading package lists... 2024-03-16T09:58:17.417 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-03-16T09:58:17.435 DEBUG:teuthology.orchestra.run.smithi017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-16T09:58:17.488 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-03-16T09:58:17.501 INFO:teuthology.orchestra.run.smithi174.stdout:Building dependency tree... 2024-03-16T09:58:17.501 INFO:teuthology.orchestra.run.smithi174.stdout:Reading state information... 2024-03-16T09:58:17.672 INFO:teuthology.orchestra.run.smithi174.stdout:linux-image-generic is already the newest version (5.15.0.100.97). 2024-03-16T09:58:17.672 INFO:teuthology.orchestra.run.smithi174.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T09:58:17.672 INFO:teuthology.orchestra.run.smithi174.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T09:58:17.672 INFO:teuthology.orchestra.run.smithi174.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T09:58:17.672 INFO:teuthology.orchestra.run.smithi174.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T09:58:17.673 INFO:teuthology.orchestra.run.smithi174.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-16T09:58:17.673 INFO:teuthology.orchestra.run.smithi174.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-16T09:58:17.673 INFO:teuthology.orchestra.run.smithi174.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T09:58:17.678 INFO:teuthology.orchestra.run.smithi017.stdout:Building dependency tree... 2024-03-16T09:58:17.679 INFO:teuthology.orchestra.run.smithi017.stdout:Reading state information... 2024-03-16T09:58:17.713 INFO:teuthology.orchestra.run.smithi174.stdout:0 upgraded, 0 newly installed, 0 to remove and 297 not upgraded. 2024-03-16T09:58:17.715 DEBUG:teuthology.orchestra.run.smithi174:> dpkg -s linux-image-generic 2024-03-16T09:58:17.731 INFO:teuthology.orchestra.run.smithi174.stdout:Package: linux-image-generic 2024-03-16T09:58:17.731 INFO:teuthology.orchestra.run.smithi174.stdout:Status: install ok installed 2024-03-16T09:58:17.731 INFO:teuthology.orchestra.run.smithi174.stdout:Priority: optional 2024-03-16T09:58:17.731 INFO:teuthology.orchestra.run.smithi174.stdout:Section: kernel 2024-03-16T09:58:17.731 INFO:teuthology.orchestra.run.smithi174.stdout:Installed-Size: 21 2024-03-16T09:58:17.731 INFO:teuthology.orchestra.run.smithi174.stdout:Maintainer: Ubuntu Kernel Team 2024-03-16T09:58:17.731 INFO:teuthology.orchestra.run.smithi174.stdout:Architecture: amd64 2024-03-16T09:58:17.732 INFO:teuthology.orchestra.run.smithi174.stdout:Source: linux-meta 2024-03-16T09:58:17.732 INFO:teuthology.orchestra.run.smithi174.stdout:Version: 5.15.0.100.97 2024-03-16T09:58:17.732 INFO:teuthology.orchestra.run.smithi174.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-16T09:58:17.732 INFO:teuthology.orchestra.run.smithi174.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-16T09:58:17.732 INFO:teuthology.orchestra.run.smithi174.stdout:Recommends: thermald 2024-03-16T09:58:17.732 INFO:teuthology.orchestra.run.smithi174.stdout:Description: Generic Linux kernel image 2024-03-16T09:58:17.732 INFO:teuthology.orchestra.run.smithi174.stdout: This package will always depend on the latest generic kernel image 2024-03-16T09:58:17.732 INFO:teuthology.orchestra.run.smithi174.stdout: available. 2024-03-16T09:58:17.732 DEBUG:teuthology.orchestra.run.smithi174:> mktemp 2024-03-16T09:58:17.779 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.PtCRzFxez1 2024-03-16T09:58:17.779 DEBUG:teuthology.orchestra.run.smithi174:> sudo cp /boot/grub/grub.cfg /tmp/tmp.PtCRzFxez1 2024-03-16T09:58:17.836 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.PtCRzFxez1 2024-03-16T09:58:17.839 INFO:teuthology.orchestra.run.smithi017.stdout:linux-image-generic is already the newest version (5.15.0.100.97). 2024-03-16T09:58:17.840 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T09:58:17.840 INFO:teuthology.orchestra.run.smithi017.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T09:58:17.840 INFO:teuthology.orchestra.run.smithi017.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T09:58:17.840 INFO:teuthology.orchestra.run.smithi017.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T09:58:17.840 INFO:teuthology.orchestra.run.smithi017.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-16T09:58:17.840 INFO:teuthology.orchestra.run.smithi017.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-16T09:58:17.840 INFO:teuthology.orchestra.run.smithi017.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T09:58:17.879 INFO:teuthology.orchestra.run.smithi017.stdout:0 upgraded, 0 newly installed, 0 to remove and 297 not upgraded. 2024-03-16T09:58:17.881 DEBUG:teuthology.orchestra.run.smithi017:> dpkg -s linux-image-generic 2024-03-16T09:58:17.942 INFO:teuthology.orchestra.run.smithi017.stdout:Package: linux-image-generic 2024-03-16T09:58:17.942 INFO:teuthology.orchestra.run.smithi017.stdout:Status: install ok installed 2024-03-16T09:58:17.942 INFO:teuthology.orchestra.run.smithi017.stdout:Priority: optional 2024-03-16T09:58:17.943 INFO:teuthology.orchestra.run.smithi017.stdout:Section: kernel 2024-03-16T09:58:17.943 INFO:teuthology.orchestra.run.smithi017.stdout:Installed-Size: 21 2024-03-16T09:58:17.943 INFO:teuthology.orchestra.run.smithi017.stdout:Maintainer: Ubuntu Kernel Team 2024-03-16T09:58:17.943 INFO:teuthology.orchestra.run.smithi017.stdout:Architecture: amd64 2024-03-16T09:58:17.943 INFO:teuthology.orchestra.run.smithi017.stdout:Source: linux-meta 2024-03-16T09:58:17.943 INFO:teuthology.orchestra.run.smithi017.stdout:Version: 5.15.0.100.97 2024-03-16T09:58:17.943 INFO:teuthology.orchestra.run.smithi017.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-16T09:58:17.943 INFO:teuthology.orchestra.run.smithi017.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-16T09:58:17.943 INFO:teuthology.orchestra.run.smithi017.stdout:Recommends: thermald 2024-03-16T09:58:17.943 INFO:teuthology.orchestra.run.smithi017.stdout:Description: Generic Linux kernel image 2024-03-16T09:58:17.943 INFO:teuthology.orchestra.run.smithi017.stdout: This package will always depend on the latest generic kernel image 2024-03-16T09:58:17.943 INFO:teuthology.orchestra.run.smithi017.stdout: available. 2024-03-16T09:58:17.944 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2024-03-16T09:58:17.950 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.PtCRzFxez1 is 10KB 2024-03-16T09:58:17.962 DEBUG:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.PtCRzFxez1 2024-03-16T09:58:17.967 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-16T09:58:17.987 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.sdAE0DQVaa 2024-03-16T09:58:17.988 DEBUG:teuthology.orchestra.run.smithi017:> sudo cp /boot/grub/grub.cfg /tmp/tmp.sdAE0DQVaa 2024-03-16T09:58:18.024 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T09:58:18.024 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-16T09:58:18.024 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-16T09:58:18.043 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.sdAE0DQVaa 2024-03-16T09:58:18.090 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-100-generic 2024-03-16T09:58:18.090 DEBUG:teuthology.orchestra.run.smithi174:> sudo update-grub 2024-03-16T09:58:18.157 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.sdAE0DQVaa is 10KB 2024-03-16T09:58:18.169 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.sdAE0DQVaa 2024-03-16T09:58:18.175 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-16T09:58:18.232 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T09:58:18.232 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-16T09:58:18.232 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-16T09:58:18.299 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-100-generic 2024-03-16T09:58:18.300 DEBUG:teuthology.orchestra.run.smithi017:> sudo update-grub 2024-03-16T09:58:19.430 INFO:teuthology.orchestra.run.smithi174.stderr:Sourcing file `/etc/default/grub' 2024-03-16T09:58:19.431 INFO:teuthology.orchestra.run.smithi174.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-16T09:58:19.440 INFO:teuthology.orchestra.run.smithi174.stderr:Generating grub configuration file ... 2024-03-16T09:58:19.461 INFO:teuthology.orchestra.run.smithi017.stderr:Sourcing file `/etc/default/grub' 2024-03-16T09:58:19.462 INFO:teuthology.orchestra.run.smithi017.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-16T09:58:19.464 INFO:teuthology.orchestra.run.smithi017.stderr:Generating grub configuration file ... 2024-03-16T09:58:19.639 INFO:teuthology.orchestra.run.smithi174.stderr:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-16T09:58:19.645 INFO:teuthology.orchestra.run.smithi174.stderr:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-16T09:58:19.649 INFO:teuthology.orchestra.run.smithi017.stderr:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-16T09:58:19.656 INFO:teuthology.orchestra.run.smithi017.stderr:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-16T09:58:19.887 INFO:teuthology.orchestra.run.smithi017.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-16T09:58:19.893 INFO:teuthology.orchestra.run.smithi017.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-16T09:58:19.913 INFO:teuthology.orchestra.run.smithi174.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-16T09:58:19.919 INFO:teuthology.orchestra.run.smithi174.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-16T09:58:19.938 INFO:teuthology.orchestra.run.smithi017.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-16T09:58:19.943 INFO:teuthology.orchestra.run.smithi017.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-16T09:58:19.961 INFO:teuthology.orchestra.run.smithi174.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-16T09:58:19.967 INFO:teuthology.orchestra.run.smithi174.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-16T09:58:20.073 INFO:teuthology.orchestra.run.smithi017.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-16T09:58:20.073 INFO:teuthology.orchestra.run.smithi017.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-16T09:58:20.073 INFO:teuthology.orchestra.run.smithi017.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-16T09:58:20.083 INFO:teuthology.orchestra.run.smithi017.stderr:done 2024-03-16T09:58:20.085 DEBUG:teuthology.orchestra.run.smithi017:> sudo shutdown -r now 2024-03-16T09:58:20.140 INFO:teuthology.orchestra.run.smithi174.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-16T09:58:20.140 INFO:teuthology.orchestra.run.smithi174.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-16T09:58:20.140 INFO:teuthology.orchestra.run.smithi174.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-16T09:58:20.152 INFO:teuthology.orchestra.run.smithi174.stderr:done 2024-03-16T09:58:20.155 DEBUG:teuthology.orchestra.run.smithi174:> sudo shutdown -r now 2024-03-16T09:58:50.114 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-16T09:58:50.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-16T09:58:50.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:58:50.158 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-16T09:58:50.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-16T09:58:50.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:59:08.574 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-03-16T09:59:17.582 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-16T09:59:17.583 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:59:20.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-03-16T09:59:32.650 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-16T09:59:32.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T09:59:50.131 DEBUG:teuthology.orchestra.remote:timed out 2024-03-16T09:59:51.070 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.174 2024-03-16T09:59:59.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-16T09:59:59.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:00:02.175 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.17 2024-03-16T10:00:06.074 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-16T10:00:06.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:00:06.421 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-03-16T10:00:06.952 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-16T10:00:06.952 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-100-generic"... 2024-03-16T10:00:06.952 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2024-03-16T10:00:06.999 INFO:teuthology.orchestra.run.smithi174.stdout:5.15.0-100-generic 2024-03-16T10:00:06.999 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-100-generic vs 5.15.0-100-generic 2024-03-16T10:00:06.999 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-16T10:00:06.999 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-16T10:00:08.000 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-16T10:00:08.000 DEBUG:teuthology.orchestra.run.smithi174:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-16T10:00:08.088 INFO:teuthology.orchestra.run.smithi174.stdout:ttyS1 2024-03-16T10:00:08.109 DEBUG:teuthology.parallel:result is None 2024-03-16T10:00:14.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-16T10:00:14.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:00:14.499 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-03-16T10:00:15.024 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-16T10:00:15.024 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-100-generic"... 2024-03-16T10:00:15.024 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2024-03-16T10:00:15.071 INFO:teuthology.orchestra.run.smithi017.stdout:5.15.0-100-generic 2024-03-16T10:00:15.071 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-100-generic vs 5.15.0-100-generic 2024-03-16T10:00:15.071 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-16T10:00:15.071 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-16T10:00:16.072 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-16T10:00:16.072 DEBUG:teuthology.orchestra.run.smithi017:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-16T10:00:16.148 INFO:teuthology.orchestra.run.smithi017.stdout:ttyS1 2024-03-16T10:00:16.169 DEBUG:teuthology.parallel:result is None 2024-03-16T10:00:16.169 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-16T10:00:16.177 INFO:teuthology.task.internal:Creating test directory... 2024-03-16T10:00:16.178 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T10:00:16.180 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T10:00:16.186 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-16T10:00:16.226 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-16T10:00:16.233 INFO:teuthology.task.internal:Creating archive directory... 2024-03-16T10:00:16.233 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T10:00:16.235 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T10:00:16.329 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-16T10:00:16.335 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-16T10:00:16.336 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T10:00:16.338 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T10:00:16.357 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T10:00:16.363 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T10:00:16.365 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T10:00:16.371 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T10:00:16.373 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-16T10:00:16.380 INFO:teuthology.task.internal:Configuring sudo... 2024-03-16T10:00:16.380 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T10:00:16.416 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T10:00:16.438 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-16T10:00:16.447 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-16T10:00:16.447 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T10:00:16.472 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T10:00:16.487 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T10:00:16.524 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T10:00:16.571 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:00:16.572 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T10:00:16.634 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T10:00:16.639 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T10:00:16.688 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:00:16.688 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T10:00:16.748 DEBUG:teuthology.orchestra.run.smithi017:> sudo service rsyslog restart 2024-03-16T10:00:16.751 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2024-03-16T10:00:16.826 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-16T10:00:16.834 INFO:teuthology.task.internal:Starting timer... 2024-03-16T10:00:16.834 INFO:teuthology.run_tasks:Running task pcp... 2024-03-16T10:00:16.844 INFO:teuthology.run_tasks:Running task selinux... 2024-03-16T10:00:16.854 DEBUG:teuthology.task.selinux:Excluding smithi017: OS 'ubuntu' does not support SELinux 2024-03-16T10:00:16.854 DEBUG:teuthology.task.selinux:Excluding smithi174: OS 'ubuntu' does not support SELinux 2024-03-16T10:00:16.855 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-16T10:00:16.855 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-16T10:00:16.855 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-16T10:00:16.855 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-16T10:00:16.864 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-16T10:00:16.880 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-16T10:00:16.881 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi017.front.sepia.ceph.com,smithi174.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-16T10:07:17.665 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi017.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com')] 2024-03-16T10:07:17.667 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-16T10:07:17.669 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:07:17.750 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-03-16T10:07:17.830 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-03-16T10:07:17.830 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-16T10:07:17.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T10:07:17.908 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-03-16T10:07:17.982 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-03-16T10:07:17.982 INFO:teuthology.run_tasks:Running task clock... 2024-03-16T10:07:17.992 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-16T10:07:17.993 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T10:07:17.994 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T10:07:17.996 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T10:07:17.996 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T10:07:18.024 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-16T10:07:18.024 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: Command line: ntpd -gq 2024-03-16T10:07:18.024 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: ---------------------------------------------------- 2024-03-16T10:07:18.025 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: ntp-4 is maintained by Network Time Foundation, 2024-03-16T10:07:18.025 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-16T10:07:18.025 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: corporation. Support and training for ntp-4 are 2024-03-16T10:07:18.025 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: available at https://www.nwtime.org/support 2024-03-16T10:07:18.025 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: ---------------------------------------------------- 2024-03-16T10:07:18.025 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: proto: precision = 0.046 usec (-24) 2024-03-16T10:07:18.025 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: basedate set to 2022-02-04 2024-03-16T10:07:18.025 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: gps base set to 2022-02-06 (week 2196) 2024-03-16T10:07:18.026 INFO:teuthology.orchestra.run.smithi017.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-16T10:07:18.026 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-16T10:07:18.026 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: restrict ::: KOD does nothing without LIMITED. 2024-03-16T10:07:18.026 INFO:teuthology.orchestra.run.smithi017.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-16T10:07:18.027 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: Listen and drop on 0 v6wildcard [::]:123 2024-03-16T10:07:18.027 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-16T10:07:18.027 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: Listen normally on 2 lo 127.0.0.1:123 2024-03-16T10:07:18.027 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: Listen normally on 3 ens1f0 172.21.15.17:123 2024-03-16T10:07:18.028 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: Listen normally on 4 lo [::1]:123 2024-03-16T10:07:18.028 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:fe59:d992%4]:123 2024-03-16T10:07:18.028 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:18 ntpd[17076]: Listening on routing socket on fd #22 for interface updates 2024-03-16T10:07:18.056 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-16T10:07:18.056 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: Command line: ntpd -gq 2024-03-16T10:07:18.056 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: ---------------------------------------------------- 2024-03-16T10:07:18.056 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: ntp-4 is maintained by Network Time Foundation, 2024-03-16T10:07:18.056 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-16T10:07:18.056 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: corporation. Support and training for ntp-4 are 2024-03-16T10:07:18.056 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: available at https://www.nwtime.org/support 2024-03-16T10:07:18.056 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: ---------------------------------------------------- 2024-03-16T10:07:18.057 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: proto: precision = 0.047 usec (-24) 2024-03-16T10:07:18.057 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: basedate set to 2022-02-04 2024-03-16T10:07:18.057 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: gps base set to 2022-02-06 (week 2196) 2024-03-16T10:07:18.057 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-16T10:07:18.057 INFO:teuthology.orchestra.run.smithi174.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-16T10:07:18.058 INFO:teuthology.orchestra.run.smithi174.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-16T10:07:18.059 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: restrict ::: KOD does nothing without LIMITED. 2024-03-16T10:07:18.059 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: Listen and drop on 0 v6wildcard [::]:123 2024-03-16T10:07:18.059 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-16T10:07:18.059 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: Listen normally on 2 lo 127.0.0.1:123 2024-03-16T10:07:18.059 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: Listen normally on 3 enp3s0f1 172.21.15.174:123 2024-03-16T10:07:18.059 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: Listen normally on 4 lo [::1]:123 2024-03-16T10:07:18.059 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:771b%5]:123 2024-03-16T10:07:18.059 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:18 ntpd[17157]: Listening on routing socket on fd #22 for interface updates 2024-03-16T10:07:19.025 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:19 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:19.025 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:19 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:19.026 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:19 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:19.026 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:19 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:19.026 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:19 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:19.026 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:19 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:19.056 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:19 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:19.056 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:19 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:19.056 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:19 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:19.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:19 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:19.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:19 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:19.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:19 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:21.026 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:21 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:21.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:21 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:21.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:21 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:21.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:21 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:21.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:21 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:21.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:21 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:21.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:21 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:21.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:21 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:21.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:21 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:21.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:21 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:21.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:21 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:21.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:21 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:21.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:21 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:21.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:21 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:21.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:21 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:21.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:21 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:21.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:21 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:21.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:21 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:21.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:21 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:21.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:21 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:21.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:21 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:21.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:21 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:21.058 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:21 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:21.058 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:21 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:21.058 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:21 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:21.058 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:21 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:22.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:22 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:22.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:22 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:22.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:22 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:22.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:22 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:22.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:22 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:22.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:22 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:23.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:23.028 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:23 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:23.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:23 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:23.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:23 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:23.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:23 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:23.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:23 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:23.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:23 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:23.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:23 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:23.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:23 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:23.058 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:23 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:23.058 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:23 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:23.058 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:23 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:23.058 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:23 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:23.058 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:23 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:23.058 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:23 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:23.058 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:23 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:24.056 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:24 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:24.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:24 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:24.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:24 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:24.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:24 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:24.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:24 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:24.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:24 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:25.026 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:25.027 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:25.028 INFO:teuthology.orchestra.run.smithi017.stdout:16 Mar 10:07:25 ntpd[17076]: ntpd: time slew +0.001968 s 2024-03-16T10:07:25.028 INFO:teuthology.orchestra.run.smithi017.stdout:ntpd: time slew +0.001968s 2024-03-16T10:07:25.028 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-16T10:07:25.028 INFO:teuthology.orchestra.run.smithi017.stderr:16 Mar 10:07:25 ntpd[17076]: can't open /var/log/ntpstats/loopstats.20240316: Permission denied 2024-03-16T10:07:25.056 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:25 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:25.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:25 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:25.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:25 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:25.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:25 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:25.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:25 ntpd[17157]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T10:07:25.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:25 ntpd[17157]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T10:07:25.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:25 ntpd[17157]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T10:07:25.057 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:25 ntpd[17157]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T10:07:25.057 INFO:teuthology.orchestra.run.smithi174.stdout:16 Mar 10:07:25 ntpd[17157]: ntpd: time slew +0.000076 s 2024-03-16T10:07:25.058 INFO:teuthology.orchestra.run.smithi174.stdout:ntpd: time slew +0.000076s 2024-03-16T10:07:25.058 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:25 ntpd[17157]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-16T10:07:25.058 INFO:teuthology.orchestra.run.smithi174.stderr:16 Mar 10:07:25 ntpd[17157]: can't open /var/log/ntpstats/loopstats.20240316: Permission denied 2024-03-16T10:07:25.076 INFO:teuthology.orchestra.run.smithi017.stdout: remote refid st t when poll reach delay offset jitter 2024-03-16T10:07:25.076 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================== 2024-03-16T10:07:25.076 INFO:teuthology.orchestra.run.smithi017.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T10:07:25.076 INFO:teuthology.orchestra.run.smithi017.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T10:07:25.076 INFO:teuthology.orchestra.run.smithi017.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T10:07:25.076 INFO:teuthology.orchestra.run.smithi017.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T10:07:25.106 INFO:teuthology.orchestra.run.smithi174.stdout: remote refid st t when poll reach delay offset jitter 2024-03-16T10:07:25.106 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================== 2024-03-16T10:07:25.106 INFO:teuthology.orchestra.run.smithi174.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T10:07:25.106 INFO:teuthology.orchestra.run.smithi174.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T10:07:25.106 INFO:teuthology.orchestra.run.smithi174.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T10:07:25.106 INFO:teuthology.orchestra.run.smithi174.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T10:07:25.107 INFO:teuthology.run_tasks:Running task install... 2024-03-16T10:07:25.116 DEBUG:teuthology.task.install:project ceph 2024-03-16T10:07:25.116 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}} 2024-03-16T10:07:25.116 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'} 2024-03-16T10:07:25.116 INFO:teuthology.task.install:Using flavor: default 2024-03-16T10:07:25.121 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-16T10:07:25.121 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-16T10:07:25.122 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-key list | grep Ceph 2024-03-16T10:07:25.122 DEBUG:teuthology.orchestra.run.smithi174:> sudo apt-key list | grep Ceph 2024-03-16T10:07:25.181 INFO:teuthology.orchestra.run.smithi017.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-16T10:07:25.207 INFO:teuthology.orchestra.run.smithi174.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-16T10:07:25.211 INFO:teuthology.orchestra.run.smithi017.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-16T10:07:25.211 INFO:teuthology.orchestra.run.smithi017.stdout:uid [ unknown] Ceph.com (release key) 2024-03-16T10:07:25.212 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-16T10:07:25.212 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-16T10:07:25.213 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:07:25.238 INFO:teuthology.orchestra.run.smithi174.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-16T10:07:25.238 INFO:teuthology.orchestra.run.smithi174.stdout:uid [ unknown] Ceph.com (release key) 2024-03-16T10:07:25.238 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-16T10:07:25.238 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-16T10:07:25.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:07:25.426 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default/ 2024-03-16T10:07:25.426 INFO:teuthology.task.install.deb:Package version is 19.0.0-2162-g3d78f5ba-1jammy 2024-03-16T10:07:25.450 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default/ 2024-03-16T10:07:25.450 INFO:teuthology.task.install.deb:Package version is 19.0.0-2162-g3d78f5ba-1jammy 2024-03-16T10:07:25.594 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:07:25.594 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-16T10:07:25.608 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-get update 2024-03-16T10:07:25.615 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:07:25.615 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-16T10:07:25.629 DEBUG:teuthology.orchestra.run.smithi174:> sudo apt-get update 2024-03-16T10:07:25.806 INFO:teuthology.orchestra.run.smithi174.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-16T10:07:25.807 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-16T10:07:25.807 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-16T10:07:25.819 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-16T10:07:25.820 INFO:teuthology.orchestra.run.smithi174.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-16T10:07:25.838 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-16T10:07:25.841 INFO:teuthology.orchestra.run.smithi174.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-16T10:07:25.843 INFO:teuthology.orchestra.run.smithi017.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy InRelease 2024-03-16T10:07:25.865 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-16T10:07:25.887 INFO:teuthology.orchestra.run.smithi017.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-16T10:07:25.906 INFO:teuthology.orchestra.run.smithi174.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy InRelease 2024-03-16T10:07:25.908 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,400 B] 2024-03-16T10:07:25.923 INFO:teuthology.orchestra.run.smithi174.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-16T10:07:25.936 INFO:teuthology.orchestra.run.smithi017.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-16T10:07:25.942 INFO:teuthology.orchestra.run.smithi174.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-16T10:07:25.980 INFO:teuthology.orchestra.run.smithi174.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-16T10:07:26.016 INFO:teuthology.orchestra.run.smithi174.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,400 B] 2024-03-16T10:07:26.058 INFO:teuthology.orchestra.run.smithi174.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-16T10:07:26.166 INFO:teuthology.orchestra.run.smithi174.stdout:Fetched 28.9 kB in 0s (79.9 kB/s) 2024-03-16T10:07:26.187 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 28.9 kB in 0s (73.7 kB/s) 2024-03-16T10:07:27.150 INFO:teuthology.orchestra.run.smithi174.stdout:Reading package lists... 2024-03-16T10:07:27.151 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-03-16T10:07:27.165 DEBUG:teuthology.orchestra.run.smithi017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2162-g3d78f5ba-1jammy cephadm=19.0.0-2162-g3d78f5ba-1jammy ceph-mds=19.0.0-2162-g3d78f5ba-1jammy ceph-mgr=19.0.0-2162-g3d78f5ba-1jammy ceph-common=19.0.0-2162-g3d78f5ba-1jammy ceph-fuse=19.0.0-2162-g3d78f5ba-1jammy ceph-test=19.0.0-2162-g3d78f5ba-1jammy ceph-volume=19.0.0-2162-g3d78f5ba-1jammy radosgw=19.0.0-2162-g3d78f5ba-1jammy python3-rados=19.0.0-2162-g3d78f5ba-1jammy python3-rgw=19.0.0-2162-g3d78f5ba-1jammy python3-cephfs=19.0.0-2162-g3d78f5ba-1jammy python3-rbd=19.0.0-2162-g3d78f5ba-1jammy libcephfs2=19.0.0-2162-g3d78f5ba-1jammy libcephfs-dev=19.0.0-2162-g3d78f5ba-1jammy librados2=19.0.0-2162-g3d78f5ba-1jammy librbd1=19.0.0-2162-g3d78f5ba-1jammy rbd-fuse=19.0.0-2162-g3d78f5ba-1jammy python3-cephfs=19.0.0-2162-g3d78f5ba-1jammy cephfs-shell=19.0.0-2162-g3d78f5ba-1jammy cephfs-top=19.0.0-2162-g3d78f5ba-1jammy cephfs-mirror=19.0.0-2162-g3d78f5ba-1jammy 2024-03-16T10:07:27.165 DEBUG:teuthology.orchestra.run.smithi174:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2162-g3d78f5ba-1jammy cephadm=19.0.0-2162-g3d78f5ba-1jammy ceph-mds=19.0.0-2162-g3d78f5ba-1jammy ceph-mgr=19.0.0-2162-g3d78f5ba-1jammy ceph-common=19.0.0-2162-g3d78f5ba-1jammy ceph-fuse=19.0.0-2162-g3d78f5ba-1jammy ceph-test=19.0.0-2162-g3d78f5ba-1jammy ceph-volume=19.0.0-2162-g3d78f5ba-1jammy radosgw=19.0.0-2162-g3d78f5ba-1jammy python3-rados=19.0.0-2162-g3d78f5ba-1jammy python3-rgw=19.0.0-2162-g3d78f5ba-1jammy python3-cephfs=19.0.0-2162-g3d78f5ba-1jammy python3-rbd=19.0.0-2162-g3d78f5ba-1jammy libcephfs2=19.0.0-2162-g3d78f5ba-1jammy libcephfs-dev=19.0.0-2162-g3d78f5ba-1jammy librados2=19.0.0-2162-g3d78f5ba-1jammy librbd1=19.0.0-2162-g3d78f5ba-1jammy rbd-fuse=19.0.0-2162-g3d78f5ba-1jammy python3-cephfs=19.0.0-2162-g3d78f5ba-1jammy cephfs-shell=19.0.0-2162-g3d78f5ba-1jammy cephfs-top=19.0.0-2162-g3d78f5ba-1jammy cephfs-mirror=19.0.0-2162-g3d78f5ba-1jammy 2024-03-16T10:07:27.217 INFO:teuthology.orchestra.run.smithi174.stdout:Reading package lists... 2024-03-16T10:07:27.228 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-03-16T10:07:27.393 INFO:teuthology.orchestra.run.smithi174.stdout:Building dependency tree... 2024-03-16T10:07:27.394 INFO:teuthology.orchestra.run.smithi174.stdout:Reading state information... 2024-03-16T10:07:27.421 INFO:teuthology.orchestra.run.smithi017.stdout:Building dependency tree... 2024-03-16T10:07:27.422 INFO:teuthology.orchestra.run.smithi017.stdout:Reading state information... 2024-03-16T10:07:27.552 INFO:teuthology.orchestra.run.smithi174.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T10:07:27.552 INFO:teuthology.orchestra.run.smithi174.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T10:07:27.552 INFO:teuthology.orchestra.run.smithi174.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T10:07:27.552 INFO:teuthology.orchestra.run.smithi174.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T10:07:27.552 INFO:teuthology.orchestra.run.smithi174.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-16T10:07:27.553 INFO:teuthology.orchestra.run.smithi174.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-16T10:07:27.553 INFO:teuthology.orchestra.run.smithi174.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T10:07:27.553 INFO:teuthology.orchestra.run.smithi174.stdout:The following additional packages will be installed: 2024-03-16T10:07:27.553 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-16T10:07:27.553 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-16T10:07:27.555 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-16T10:07:27.556 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-16T10:07:27.556 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-16T10:07:27.556 INFO:teuthology.orchestra.run.smithi174.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-16T10:07:27.556 INFO:teuthology.orchestra.run.smithi174.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-16T10:07:27.556 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-16T10:07:27.556 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-16T10:07:27.556 INFO:teuthology.orchestra.run.smithi174.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-16T10:07:27.556 INFO:teuthology.orchestra.run.smithi174.stdout:Suggested packages: 2024-03-16T10:07:27.556 INFO:teuthology.orchestra.run.smithi174.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-16T10:07:27.556 INFO:teuthology.orchestra.run.smithi174.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-16T10:07:27.556 INFO:teuthology.orchestra.run.smithi174.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-16T10:07:27.557 INFO:teuthology.orchestra.run.smithi174.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-16T10:07:27.557 INFO:teuthology.orchestra.run.smithi174.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-16T10:07:27.557 INFO:teuthology.orchestra.run.smithi174.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-16T10:07:27.557 INFO:teuthology.orchestra.run.smithi174.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-16T10:07:27.557 INFO:teuthology.orchestra.run.smithi174.stdout:Recommended packages: 2024-03-16T10:07:27.557 INFO:teuthology.orchestra.run.smithi174.stdout: btrfs-tools 2024-03-16T10:07:27.610 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T10:07:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T10:07:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T10:07:27.610 INFO:teuthology.orchestra.run.smithi017.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T10:07:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-16T10:07:27.611 INFO:teuthology.orchestra.run.smithi017.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-16T10:07:27.611 INFO:teuthology.orchestra.run.smithi017.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T10:07:27.611 INFO:teuthology.orchestra.run.smithi017.stdout:The following additional packages will be installed: 2024-03-16T10:07:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-16T10:07:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-16T10:07:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-16T10:07:27.612 INFO:teuthology.orchestra.run.smithi017.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-16T10:07:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-16T10:07:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-16T10:07:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-16T10:07:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-16T10:07:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-16T10:07:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-16T10:07:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-16T10:07:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-16T10:07:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-16T10:07:27.613 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-16T10:07:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-16T10:07:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-16T10:07:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-16T10:07:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-16T10:07:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-16T10:07:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-16T10:07:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-16T10:07:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-16T10:07:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-16T10:07:27.614 INFO:teuthology.orchestra.run.smithi017.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-16T10:07:27.615 INFO:teuthology.orchestra.run.smithi017.stdout:Suggested packages: 2024-03-16T10:07:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-16T10:07:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-16T10:07:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-16T10:07:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-16T10:07:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-16T10:07:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-16T10:07:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-16T10:07:27.615 INFO:teuthology.orchestra.run.smithi017.stdout:Recommended packages: 2024-03-16T10:07:27.615 INFO:teuthology.orchestra.run.smithi017.stdout: btrfs-tools 2024-03-16T10:07:27.663 INFO:teuthology.orchestra.run.smithi174.stdout:The following NEW packages will be installed: 2024-03-16T10:07:27.663 INFO:teuthology.orchestra.run.smithi174.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-16T10:07:27.663 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-16T10:07:27.664 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-16T10:07:27.664 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-16T10:07:27.665 INFO:teuthology.orchestra.run.smithi174.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-16T10:07:27.665 INFO:teuthology.orchestra.run.smithi174.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-16T10:07:27.665 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-16T10:07:27.665 INFO:teuthology.orchestra.run.smithi174.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-16T10:07:27.665 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-16T10:07:27.665 INFO:teuthology.orchestra.run.smithi174.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-16T10:07:27.665 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-16T10:07:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-16T10:07:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-16T10:07:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-16T10:07:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-16T10:07:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-16T10:07:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-16T10:07:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-16T10:07:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-16T10:07:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-16T10:07:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-16T10:07:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-16T10:07:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-16T10:07:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-16T10:07:27.666 INFO:teuthology.orchestra.run.smithi174.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-16T10:07:27.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-16T10:07:27.667 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-16T10:07:27.667 INFO:teuthology.orchestra.run.smithi174.stdout: socat unzip xmlstarlet zip 2024-03-16T10:07:27.726 INFO:teuthology.orchestra.run.smithi017.stdout:The following NEW packages will be installed: 2024-03-16T10:07:27.727 INFO:teuthology.orchestra.run.smithi017.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-16T10:07:27.727 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-16T10:07:27.727 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-16T10:07:27.727 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-16T10:07:27.728 INFO:teuthology.orchestra.run.smithi017.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-16T10:07:27.728 INFO:teuthology.orchestra.run.smithi017.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-16T10:07:27.728 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-16T10:07:27.728 INFO:teuthology.orchestra.run.smithi017.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-16T10:07:27.728 INFO:teuthology.orchestra.run.smithi017.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-16T10:07:27.728 INFO:teuthology.orchestra.run.smithi017.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-16T10:07:27.729 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-16T10:07:27.730 INFO:teuthology.orchestra.run.smithi017.stdout: socat unzip xmlstarlet zip 2024-03-16T10:07:27.794 INFO:teuthology.orchestra.run.smithi017.stdout:0 upgraded, 124 newly installed, 0 to remove and 284 not upgraded. 2024-03-16T10:07:27.794 INFO:teuthology.orchestra.run.smithi017.stdout:Need to get 178 MB of archives. 2024-03-16T10:07:27.794 INFO:teuthology.orchestra.run.smithi017.stdout:After this operation, 739 MB of additional disk space will be used. 2024-03-16T10:07:27.794 INFO:teuthology.orchestra.run.smithi017.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-16T10:07:27.815 INFO:teuthology.orchestra.run.smithi017.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-16T10:07:27.826 INFO:teuthology.orchestra.run.smithi017.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-16T10:07:27.853 INFO:teuthology.orchestra.run.smithi174.stdout:0 upgraded, 124 newly installed, 0 to remove and 284 not upgraded. 2024-03-16T10:07:27.853 INFO:teuthology.orchestra.run.smithi174.stdout:Need to get 178 MB of archives. 2024-03-16T10:07:27.853 INFO:teuthology.orchestra.run.smithi174.stdout:After this operation, 739 MB of additional disk space will be used. 2024-03-16T10:07:27.853 INFO:teuthology.orchestra.run.smithi174.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-16T10:07:27.872 INFO:teuthology.orchestra.run.smithi017.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-16T10:07:27.876 INFO:teuthology.orchestra.run.smithi017.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2162-g3d78f5ba-1jammy [3,554 kB] 2024-03-16T10:07:27.883 INFO:teuthology.orchestra.run.smithi174.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2162-g3d78f5ba-1jammy [3,554 kB] 2024-03-16T10:07:27.898 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-16T10:07:27.899 INFO:teuthology.orchestra.run.smithi017.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-16T10:07:27.912 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-16T10:07:27.936 INFO:teuthology.orchestra.run.smithi174.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-16T10:07:27.968 INFO:teuthology.orchestra.run.smithi017.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-16T10:07:27.973 INFO:teuthology.orchestra.run.smithi017.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-16T10:07:27.982 INFO:teuthology.orchestra.run.smithi174.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-16T10:07:27.984 INFO:teuthology.orchestra.run.smithi017.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-16T10:07:27.987 INFO:teuthology.orchestra.run.smithi017.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-16T10:07:27.988 INFO:teuthology.orchestra.run.smithi017.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-16T10:07:27.996 INFO:teuthology.orchestra.run.smithi017.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-16T10:07:27.998 INFO:teuthology.orchestra.run.smithi017.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-16T10:07:28.000 INFO:teuthology.orchestra.run.smithi017.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-16T10:07:28.002 INFO:teuthology.orchestra.run.smithi017.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-16T10:07:28.002 INFO:teuthology.orchestra.run.smithi017.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-16T10:07:28.011 INFO:teuthology.orchestra.run.smithi017.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-16T10:07:28.014 INFO:teuthology.orchestra.run.smithi017.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-16T10:07:28.016 INFO:teuthology.orchestra.run.smithi017.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-16T10:07:28.016 INFO:teuthology.orchestra.run.smithi017.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-16T10:07:28.031 INFO:teuthology.orchestra.run.smithi017.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-16T10:07:28.032 INFO:teuthology.orchestra.run.smithi017.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-16T10:07:28.032 INFO:teuthology.orchestra.run.smithi017.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-16T10:07:28.033 INFO:teuthology.orchestra.run.smithi017.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-16T10:07:28.033 INFO:teuthology.orchestra.run.smithi017.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-16T10:07:28.033 INFO:teuthology.orchestra.run.smithi017.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-16T10:07:28.050 INFO:teuthology.orchestra.run.smithi017.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-16T10:07:28.051 INFO:teuthology.orchestra.run.smithi017.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-16T10:07:28.054 INFO:teuthology.orchestra.run.smithi017.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-16T10:07:28.054 INFO:teuthology.orchestra.run.smithi017.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-16T10:07:28.070 INFO:teuthology.orchestra.run.smithi017.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-16T10:07:28.071 INFO:teuthology.orchestra.run.smithi017.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-16T10:07:28.071 INFO:teuthology.orchestra.run.smithi017.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-16T10:07:28.072 INFO:teuthology.orchestra.run.smithi017.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-16T10:07:28.073 INFO:teuthology.orchestra.run.smithi017.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-16T10:07:28.073 INFO:teuthology.orchestra.run.smithi017.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-16T10:07:28.090 INFO:teuthology.orchestra.run.smithi017.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-16T10:07:28.090 INFO:teuthology.orchestra.run.smithi017.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-16T10:07:28.097 INFO:teuthology.orchestra.run.smithi017.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-16T10:07:28.097 INFO:teuthology.orchestra.run.smithi017.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-16T10:07:28.109 INFO:teuthology.orchestra.run.smithi017.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-16T10:07:28.110 INFO:teuthology.orchestra.run.smithi017.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-16T10:07:28.113 INFO:teuthology.orchestra.run.smithi017.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-16T10:07:28.116 INFO:teuthology.orchestra.run.smithi017.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-16T10:07:28.116 INFO:teuthology.orchestra.run.smithi017.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-16T10:07:28.117 INFO:teuthology.orchestra.run.smithi017.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-16T10:07:28.128 INFO:teuthology.orchestra.run.smithi017.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-16T10:07:28.131 INFO:teuthology.orchestra.run.smithi017.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-16T10:07:28.132 INFO:teuthology.orchestra.run.smithi017.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-16T10:07:28.132 INFO:teuthology.orchestra.run.smithi017.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-16T10:07:28.140 INFO:teuthology.orchestra.run.smithi017.stdout:Get:53 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2162-g3d78f5ba-1jammy [3,225 kB] 2024-03-16T10:07:28.148 INFO:teuthology.orchestra.run.smithi017.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-16T10:07:28.185 INFO:teuthology.orchestra.run.smithi174.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-16T10:07:28.203 INFO:teuthology.orchestra.run.smithi017.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-16T10:07:28.204 INFO:teuthology.orchestra.run.smithi017.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-16T10:07:28.205 INFO:teuthology.orchestra.run.smithi017.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-16T10:07:28.206 INFO:teuthology.orchestra.run.smithi017.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-16T10:07:28.207 INFO:teuthology.orchestra.run.smithi017.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-16T10:07:28.227 INFO:teuthology.orchestra.run.smithi017.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-16T10:07:28.229 INFO:teuthology.orchestra.run.smithi017.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-16T10:07:28.229 INFO:teuthology.orchestra.run.smithi017.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-16T10:07:28.245 INFO:teuthology.orchestra.run.smithi017.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-16T10:07:28.245 INFO:teuthology.orchestra.run.smithi017.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-16T10:07:28.254 INFO:teuthology.orchestra.run.smithi017.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-16T10:07:28.254 INFO:teuthology.orchestra.run.smithi017.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-16T10:07:28.255 INFO:teuthology.orchestra.run.smithi017.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-16T10:07:28.255 INFO:teuthology.orchestra.run.smithi017.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-16T10:07:28.256 INFO:teuthology.orchestra.run.smithi017.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-16T10:07:28.256 INFO:teuthology.orchestra.run.smithi017.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-16T10:07:28.259 INFO:teuthology.orchestra.run.smithi017.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-16T10:07:28.273 INFO:teuthology.orchestra.run.smithi017.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-16T10:07:28.275 INFO:teuthology.orchestra.run.smithi017.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-16T10:07:28.275 INFO:teuthology.orchestra.run.smithi017.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-16T10:07:28.280 INFO:teuthology.orchestra.run.smithi174.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-16T10:07:28.285 INFO:teuthology.orchestra.run.smithi017.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2162-g3d78f5ba-1jammy [712 kB] 2024-03-16T10:07:28.292 INFO:teuthology.orchestra.run.smithi017.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-16T10:07:28.293 INFO:teuthology.orchestra.run.smithi174.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-16T10:07:28.295 INFO:teuthology.orchestra.run.smithi017.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-16T10:07:28.295 INFO:teuthology.orchestra.run.smithi017.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-16T10:07:28.296 INFO:teuthology.orchestra.run.smithi017.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-16T10:07:28.297 INFO:teuthology.orchestra.run.smithi017.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-16T10:07:28.301 INFO:teuthology.orchestra.run.smithi017.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-16T10:07:28.302 INFO:teuthology.orchestra.run.smithi017.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-16T10:07:28.308 INFO:teuthology.orchestra.run.smithi017.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2162-g3d78f5ba-1jammy [357 kB] 2024-03-16T10:07:28.312 INFO:teuthology.orchestra.run.smithi017.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-16T10:07:28.312 INFO:teuthology.orchestra.run.smithi017.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-16T10:07:28.313 INFO:teuthology.orchestra.run.smithi017.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-16T10:07:28.319 INFO:teuthology.orchestra.run.smithi017.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2162-g3d78f5ba-1jammy [32.8 kB] 2024-03-16T10:07:28.320 INFO:teuthology.orchestra.run.smithi017.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2162-g3d78f5ba-1jammy [180 kB] 2024-03-16T10:07:28.326 INFO:teuthology.orchestra.run.smithi017.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2162-g3d78f5ba-1jammy [64.6 kB] 2024-03-16T10:07:28.328 INFO:teuthology.orchestra.run.smithi017.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2162-g3d78f5ba-1jammy [330 kB] 2024-03-16T10:07:28.332 INFO:teuthology.orchestra.run.smithi017.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-16T10:07:28.333 INFO:teuthology.orchestra.run.smithi017.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-16T10:07:28.334 INFO:teuthology.orchestra.run.smithi017.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-16T10:07:28.342 INFO:teuthology.orchestra.run.smithi017.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-16T10:07:28.342 INFO:teuthology.orchestra.run.smithi017.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-16T10:07:28.342 INFO:teuthology.orchestra.run.smithi017.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-16T10:07:28.344 INFO:teuthology.orchestra.run.smithi017.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-16T10:07:28.345 INFO:teuthology.orchestra.run.smithi174.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-16T10:07:28.349 INFO:teuthology.orchestra.run.smithi174.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2162-g3d78f5ba-1jammy [3,225 kB] 2024-03-16T10:07:28.351 INFO:teuthology.orchestra.run.smithi017.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-16T10:07:28.357 INFO:teuthology.orchestra.run.smithi017.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2162-g3d78f5ba-1jammy [6,653 kB] 2024-03-16T10:07:28.519 INFO:teuthology.orchestra.run.smithi174.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-16T10:07:28.528 INFO:teuthology.orchestra.run.smithi174.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-16T10:07:28.556 INFO:teuthology.orchestra.run.smithi174.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-16T10:07:28.564 INFO:teuthology.orchestra.run.smithi174.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-16T10:07:28.665 INFO:teuthology.orchestra.run.smithi174.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2162-g3d78f5ba-1jammy [712 kB] 2024-03-16T10:07:28.684 INFO:teuthology.orchestra.run.smithi017.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2162-g3d78f5ba-1jammy [112 kB] 2024-03-16T10:07:28.687 INFO:teuthology.orchestra.run.smithi017.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2162-g3d78f5ba-1jammy [466 kB] 2024-03-16T10:07:28.704 INFO:teuthology.orchestra.run.smithi017.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2162-g3d78f5ba-1jammy [25.3 MB] 2024-03-16T10:07:28.729 INFO:teuthology.orchestra.run.smithi174.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-16T10:07:28.740 INFO:teuthology.orchestra.run.smithi174.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2162-g3d78f5ba-1jammy [357 kB] 2024-03-16T10:07:28.768 INFO:teuthology.orchestra.run.smithi174.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2162-g3d78f5ba-1jammy [32.8 kB] 2024-03-16T10:07:28.773 INFO:teuthology.orchestra.run.smithi174.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2162-g3d78f5ba-1jammy [180 kB] 2024-03-16T10:07:28.795 INFO:teuthology.orchestra.run.smithi174.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2162-g3d78f5ba-1jammy [64.6 kB] 2024-03-16T10:07:28.801 INFO:teuthology.orchestra.run.smithi174.stdout:Get:20 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2162-g3d78f5ba-1jammy [330 kB] 2024-03-16T10:07:28.825 INFO:teuthology.orchestra.run.smithi174.stdout:Get:21 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2162-g3d78f5ba-1jammy [6,653 kB] 2024-03-16T10:07:29.294 INFO:teuthology.orchestra.run.smithi174.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-16T10:07:29.309 INFO:teuthology.orchestra.run.smithi174.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-16T10:07:29.325 INFO:teuthology.orchestra.run.smithi174.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-16T10:07:29.333 INFO:teuthology.orchestra.run.smithi174.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-16T10:07:29.334 INFO:teuthology.orchestra.run.smithi174.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-16T10:07:29.349 INFO:teuthology.orchestra.run.smithi174.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-16T10:07:29.387 INFO:teuthology.orchestra.run.smithi174.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-16T10:07:29.400 INFO:teuthology.orchestra.run.smithi174.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-16T10:07:29.404 INFO:teuthology.orchestra.run.smithi174.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-16T10:07:29.405 INFO:teuthology.orchestra.run.smithi174.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-16T10:07:29.446 INFO:teuthology.orchestra.run.smithi174.stdout:Get:32 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2162-g3d78f5ba-1jammy [112 kB] 2024-03-16T10:07:29.454 INFO:teuthology.orchestra.run.smithi174.stdout:Get:33 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2162-g3d78f5ba-1jammy [466 kB] 2024-03-16T10:07:29.502 INFO:teuthology.orchestra.run.smithi174.stdout:Get:34 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2162-g3d78f5ba-1jammy [25.3 MB] 2024-03-16T10:07:29.576 INFO:teuthology.orchestra.run.smithi174.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-16T10:07:29.659 INFO:teuthology.orchestra.run.smithi174.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-16T10:07:29.677 INFO:teuthology.orchestra.run.smithi174.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-16T10:07:29.741 INFO:teuthology.orchestra.run.smithi174.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-16T10:07:29.774 INFO:teuthology.orchestra.run.smithi174.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-16T10:07:29.794 INFO:teuthology.orchestra.run.smithi174.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-16T10:07:29.819 INFO:teuthology.orchestra.run.smithi174.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-16T10:07:30.002 INFO:teuthology.orchestra.run.smithi174.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-16T10:07:30.018 INFO:teuthology.orchestra.run.smithi174.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-16T10:07:30.025 INFO:teuthology.orchestra.run.smithi174.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-16T10:07:30.030 INFO:teuthology.orchestra.run.smithi174.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-16T10:07:30.067 INFO:teuthology.orchestra.run.smithi017.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2162-g3d78f5ba-1jammy [5,085 kB] 2024-03-16T10:07:30.213 INFO:teuthology.orchestra.run.smithi174.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-16T10:07:30.230 INFO:teuthology.orchestra.run.smithi017.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2162-g3d78f5ba-1jammy [244 kB] 2024-03-16T10:07:30.242 INFO:teuthology.orchestra.run.smithi017.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2162-g3d78f5ba-1jammy [124 kB] 2024-03-16T10:07:30.247 INFO:teuthology.orchestra.run.smithi017.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2162-g3d78f5ba-1jammy [1,526 kB] 2024-03-16T10:07:30.296 INFO:teuthology.orchestra.run.smithi174.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-16T10:07:30.310 INFO:teuthology.orchestra.run.smithi174.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-16T10:07:30.324 INFO:teuthology.orchestra.run.smithi017.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2162-g3d78f5ba-1jammy [6,255 kB] 2024-03-16T10:07:30.489 INFO:teuthology.orchestra.run.smithi174.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-16T10:07:30.534 INFO:teuthology.orchestra.run.smithi174.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-16T10:07:30.549 INFO:teuthology.orchestra.run.smithi174.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-16T10:07:30.637 INFO:teuthology.orchestra.run.smithi017.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2162-g3d78f5ba-1jammy [22.8 MB] 2024-03-16T10:07:30.709 INFO:teuthology.orchestra.run.smithi174.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-16T10:07:30.710 INFO:teuthology.orchestra.run.smithi174.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-16T10:07:30.715 INFO:teuthology.orchestra.run.smithi174.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-16T10:07:30.719 INFO:teuthology.orchestra.run.smithi174.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-16T10:07:30.734 INFO:teuthology.orchestra.run.smithi174.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-16T10:07:30.919 INFO:teuthology.orchestra.run.smithi174.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-16T10:07:31.176 INFO:teuthology.orchestra.run.smithi174.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-16T10:07:31.290 INFO:teuthology.orchestra.run.smithi174.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-16T10:07:31.333 INFO:teuthology.orchestra.run.smithi174.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-16T10:07:31.417 INFO:teuthology.orchestra.run.smithi174.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-16T10:07:31.429 INFO:teuthology.orchestra.run.smithi174.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-16T10:07:31.442 INFO:teuthology.orchestra.run.smithi174.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-16T10:07:31.482 INFO:teuthology.orchestra.run.smithi174.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-16T10:07:31.745 INFO:teuthology.orchestra.run.smithi174.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-16T10:07:31.747 INFO:teuthology.orchestra.run.smithi174.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-16T10:07:31.749 INFO:teuthology.orchestra.run.smithi174.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-16T10:07:31.879 INFO:teuthology.orchestra.run.smithi017.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2162-g3d78f5ba-1jammy [14.1 kB] 2024-03-16T10:07:31.880 INFO:teuthology.orchestra.run.smithi017.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2162-g3d78f5ba-1jammy [870 kB] 2024-03-16T10:07:31.896 INFO:teuthology.orchestra.run.smithi017.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2162-g3d78f5ba-1jammy [2,434 kB] 2024-03-16T10:07:31.913 INFO:teuthology.orchestra.run.smithi174.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-16T10:07:31.942 INFO:teuthology.orchestra.run.smithi174.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2162-g3d78f5ba-1jammy [5,085 kB] 2024-03-16T10:07:31.956 INFO:teuthology.orchestra.run.smithi017.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2162-g3d78f5ba-1jammy [777 kB] 2024-03-16T10:07:31.986 INFO:teuthology.orchestra.run.smithi017.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2162-g3d78f5ba-1jammy [144 kB] 2024-03-16T10:07:31.990 INFO:teuthology.orchestra.run.smithi017.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2162-g3d78f5ba-1jammy [1,936 kB] 2024-03-16T10:07:32.083 INFO:teuthology.orchestra.run.smithi017.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2162-g3d78f5ba-1jammy [8,625 kB] 2024-03-16T10:07:32.252 INFO:teuthology.orchestra.run.smithi174.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-16T10:07:32.371 INFO:teuthology.orchestra.run.smithi174.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2162-g3d78f5ba-1jammy [244 kB] 2024-03-16T10:07:32.386 INFO:teuthology.orchestra.run.smithi174.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2162-g3d78f5ba-1jammy [124 kB] 2024-03-16T10:07:32.406 INFO:teuthology.orchestra.run.smithi174.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2162-g3d78f5ba-1jammy [1,526 kB] 2024-03-16T10:07:32.515 INFO:teuthology.orchestra.run.smithi174.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2162-g3d78f5ba-1jammy [6,255 kB] 2024-03-16T10:07:32.552 INFO:teuthology.orchestra.run.smithi017.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2162-g3d78f5ba-1jammy [14.2 kB] 2024-03-16T10:07:32.553 INFO:teuthology.orchestra.run.smithi017.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2162-g3d78f5ba-1jammy [50.2 MB] 2024-03-16T10:07:32.649 INFO:teuthology.orchestra.run.smithi174.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-16T10:07:32.658 INFO:teuthology.orchestra.run.smithi174.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-16T10:07:32.660 INFO:teuthology.orchestra.run.smithi174.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-16T10:07:32.728 INFO:teuthology.orchestra.run.smithi174.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-16T10:07:32.729 INFO:teuthology.orchestra.run.smithi174.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-16T10:07:32.730 INFO:teuthology.orchestra.run.smithi174.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-16T10:07:32.732 INFO:teuthology.orchestra.run.smithi174.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-16T10:07:32.733 INFO:teuthology.orchestra.run.smithi174.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-16T10:07:32.898 INFO:teuthology.orchestra.run.smithi174.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-16T10:07:32.985 INFO:teuthology.orchestra.run.smithi174.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-16T10:07:33.015 INFO:teuthology.orchestra.run.smithi174.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2162-g3d78f5ba-1jammy [22.8 MB] 2024-03-16T10:07:33.082 INFO:teuthology.orchestra.run.smithi174.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-16T10:07:33.327 INFO:teuthology.orchestra.run.smithi174.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-16T10:07:33.370 INFO:teuthology.orchestra.run.smithi174.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-16T10:07:33.394 INFO:teuthology.orchestra.run.smithi174.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-16T10:07:33.403 INFO:teuthology.orchestra.run.smithi174.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-16T10:07:33.447 INFO:teuthology.orchestra.run.smithi174.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-16T10:07:33.473 INFO:teuthology.orchestra.run.smithi174.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-16T10:07:33.474 INFO:teuthology.orchestra.run.smithi174.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-16T10:07:33.481 INFO:teuthology.orchestra.run.smithi174.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-16T10:07:33.647 INFO:teuthology.orchestra.run.smithi174.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-16T10:07:34.186 INFO:teuthology.orchestra.run.smithi174.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-16T10:07:34.192 INFO:teuthology.orchestra.run.smithi174.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-16T10:07:34.230 INFO:teuthology.orchestra.run.smithi174.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-16T10:07:34.231 INFO:teuthology.orchestra.run.smithi174.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-16T10:07:34.237 INFO:teuthology.orchestra.run.smithi174.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-16T10:07:34.240 INFO:teuthology.orchestra.run.smithi174.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-16T10:07:34.281 INFO:teuthology.orchestra.run.smithi174.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-16T10:07:34.290 INFO:teuthology.orchestra.run.smithi174.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-16T10:07:34.362 INFO:teuthology.orchestra.run.smithi174.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-16T10:07:34.363 INFO:teuthology.orchestra.run.smithi174.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-16T10:07:34.528 INFO:teuthology.orchestra.run.smithi174.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-16T10:07:34.875 INFO:teuthology.orchestra.run.smithi174.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2162-g3d78f5ba-1jammy [14.1 kB] 2024-03-16T10:07:34.876 INFO:teuthology.orchestra.run.smithi174.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2162-g3d78f5ba-1jammy [870 kB] 2024-03-16T10:07:34.893 INFO:teuthology.orchestra.run.smithi174.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2162-g3d78f5ba-1jammy [2,434 kB] 2024-03-16T10:07:34.970 INFO:teuthology.orchestra.run.smithi174.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-16T10:07:34.985 INFO:teuthology.orchestra.run.smithi174.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-16T10:07:35.068 INFO:teuthology.orchestra.run.smithi174.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2162-g3d78f5ba-1jammy [777 kB] 2024-03-16T10:07:35.128 INFO:teuthology.orchestra.run.smithi174.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2162-g3d78f5ba-1jammy [144 kB] 2024-03-16T10:07:35.141 INFO:teuthology.orchestra.run.smithi174.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2162-g3d78f5ba-1jammy [1,936 kB] 2024-03-16T10:07:35.293 INFO:teuthology.orchestra.run.smithi174.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2162-g3d78f5ba-1jammy [8,625 kB] 2024-03-16T10:07:35.622 INFO:teuthology.orchestra.run.smithi017.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2162-g3d78f5ba-1jammy [121 kB] 2024-03-16T10:07:35.624 INFO:teuthology.orchestra.run.smithi017.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2162-g3d78f5ba-1jammy [230 kB] 2024-03-16T10:07:35.629 INFO:teuthology.orchestra.run.smithi017.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2162-g3d78f5ba-1jammy [34.0 kB] 2024-03-16T10:07:35.630 INFO:teuthology.orchestra.run.smithi017.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2162-g3d78f5ba-1jammy [23.4 kB] 2024-03-16T10:07:35.631 INFO:teuthology.orchestra.run.smithi017.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2162-g3d78f5ba-1jammy [39.0 kB] 2024-03-16T10:07:35.632 INFO:teuthology.orchestra.run.smithi017.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2162-g3d78f5ba-1jammy [13.1 MB] 2024-03-16T10:07:35.904 INFO:teuthology.orchestra.run.smithi174.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2162-g3d78f5ba-1jammy [14.2 kB] 2024-03-16T10:07:35.904 INFO:teuthology.orchestra.run.smithi174.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2162-g3d78f5ba-1jammy [50.2 MB] 2024-03-16T10:07:36.184 INFO:teuthology.orchestra.run.smithi017.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2162-g3d78f5ba-1jammy [91.5 kB] 2024-03-16T10:07:36.818 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 178 MB in 8s (21.1 MB/s) 2024-03-16T10:07:36.960 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-16T10:07:37.061 INFO:teuthology.orchestra.run.smithi017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-16T10:07:37.067 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-16T10:07:37.108 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-16T10:07:37.402 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-16T10:07:37.429 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-16T10:07:37.469 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-16T10:07:37.788 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cmd2. 2024-03-16T10:07:37.815 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-16T10:07:37.855 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-16T10:07:38.043 INFO:teuthology.orchestra.run.smithi174.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2162-g3d78f5ba-1jammy [121 kB] 2024-03-16T10:07:38.045 INFO:teuthology.orchestra.run.smithi174.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2162-g3d78f5ba-1jammy [230 kB] 2024-03-16T10:07:38.050 INFO:teuthology.orchestra.run.smithi174.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2162-g3d78f5ba-1jammy [34.0 kB] 2024-03-16T10:07:38.051 INFO:teuthology.orchestra.run.smithi174.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2162-g3d78f5ba-1jammy [23.4 kB] 2024-03-16T10:07:38.052 INFO:teuthology.orchestra.run.smithi174.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2162-g3d78f5ba-1jammy [39.0 kB] 2024-03-16T10:07:38.053 INFO:teuthology.orchestra.run.smithi174.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2162-g3d78f5ba-1jammy [13.1 MB] 2024-03-16T10:07:38.199 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-16T10:07:38.226 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-16T10:07:38.308 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-16T10:07:38.410 INFO:teuthology.orchestra.run.smithi174.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2162-g3d78f5ba-1jammy [91.5 kB] 2024-03-16T10:07:38.661 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-16T10:07:38.688 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-16T10:07:38.728 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-16T10:07:39.013 INFO:teuthology.orchestra.run.smithi174.stdout:Fetched 178 MB in 11s (16.6 MB/s) 2024-03-16T10:07:39.030 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-16T10:07:39.057 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-16T10:07:39.089 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-16T10:07:39.128 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-16T10:07:39.220 INFO:teuthology.orchestra.run.smithi174.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-16T10:07:39.225 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-16T10:07:39.259 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-16T10:07:39.534 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-16T10:07:39.537 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-16T10:07:39.553 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-16T10:07:39.561 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-16T10:07:39.595 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-16T10:07:39.617 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T10:07:39.915 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-cmd2. 2024-03-16T10:07:39.941 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-16T10:07:39.981 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-16T10:07:40.112 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-16T10:07:40.139 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-16T10:07:40.187 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T10:07:40.334 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-16T10:07:40.361 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-16T10:07:40.443 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-16T10:07:40.748 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-16T10:07:40.771 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-16T10:07:40.775 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-16T10:07:40.797 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-16T10:07:40.814 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T10:07:40.837 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-16T10:07:41.157 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-16T10:07:41.159 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-16T10:07:41.183 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-16T10:07:41.186 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-16T10:07:41.224 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-16T10:07:41.226 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-16T10:07:41.537 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librados2. 2024-03-16T10:07:41.564 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../010-librados2_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:41.604 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librados2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:41.677 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-16T10:07:41.704 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-16T10:07:41.752 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T10:07:42.032 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librbd1. 2024-03-16T10:07:42.058 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../011-librbd1_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:42.098 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librbd1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:42.122 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-16T10:07:42.137 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-16T10:07:42.172 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T10:07:42.516 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-16T10:07:42.543 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-16T10:07:42.571 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libcephfs2. 2024-03-16T10:07:42.583 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T10:07:42.598 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:42.642 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libcephfs2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:42.920 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rados. 2024-03-16T10:07:42.944 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-16T10:07:42.946 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:42.971 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-16T10:07:42.986 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rados (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:43.011 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-16T10:07:43.281 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-16T10:07:43.308 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:07:43.330 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package librados2. 2024-03-16T10:07:43.339 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-ceph-argparse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:43.357 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../010-librados2_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:43.397 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking librados2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:43.558 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cephfs. 2024-03-16T10:07:43.585 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:43.624 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cephfs (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:43.791 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package librbd1. 2024-03-16T10:07:43.806 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../011-librbd1_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:43.858 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking librbd1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:43.952 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-16T10:07:43.979 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:07:44.019 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:44.311 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libcephfs2. 2024-03-16T10:07:44.337 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:44.338 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-prettytable. 2024-03-16T10:07:44.365 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-16T10:07:44.377 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libcephfs2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:44.405 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-16T10:07:44.663 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-rados. 2024-03-16T10:07:44.666 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rbd. 2024-03-16T10:07:44.678 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:44.693 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:44.721 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-rados (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:44.732 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rbd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:45.024 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-16T10:07:45.035 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-16T10:07:45.050 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:07:45.063 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-16T10:07:45.090 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-ceph-argparse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:45.093 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-16T10:07:45.326 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-cephfs. 2024-03-16T10:07:45.352 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:45.354 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-16T10:07:45.370 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-16T10:07:45.393 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-cephfs (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:45.412 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-16T10:07:45.707 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-16T10:07:45.712 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-16T10:07:45.734 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-16T10:07:45.738 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:07:45.773 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-16T10:07:45.804 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:46.115 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-prettytable. 2024-03-16T10:07:46.119 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package lua5.1. 2024-03-16T10:07:46.141 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-16T10:07:46.146 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-16T10:07:46.181 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-16T10:07:46.185 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-16T10:07:46.480 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package lua-any. 2024-03-16T10:07:46.496 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-16T10:07:46.538 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-16T10:07:46.651 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-rbd. 2024-03-16T10:07:46.678 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:46.717 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-rbd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:46.807 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package zip. 2024-03-16T10:07:46.834 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-16T10:07:46.874 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking zip (3.0-12build2) ... 2024-03-16T10:07:46.995 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-16T10:07:47.010 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-16T10:07:47.045 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-16T10:07:47.210 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package unzip. 2024-03-16T10:07:47.237 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-16T10:07:47.277 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-16T10:07:47.323 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-16T10:07:47.338 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-16T10:07:47.381 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-16T10:07:47.664 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package luarocks. 2024-03-16T10:07:47.690 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-16T10:07:47.692 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-16T10:07:47.709 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-16T10:07:47.730 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-16T10:07:47.750 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-16T10:07:48.095 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package lua5.1. 2024-03-16T10:07:48.096 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librgw2. 2024-03-16T10:07:48.122 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-16T10:07:48.123 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../027-librgw2_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:48.162 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-16T10:07:48.162 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librgw2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:48.473 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package lua-any. 2024-03-16T10:07:48.500 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-16T10:07:48.539 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-16T10:07:48.757 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rgw. 2024-03-16T10:07:48.784 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:48.800 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package zip. 2024-03-16T10:07:48.827 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-16T10:07:48.831 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:48.875 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking zip (3.0-12build2) ... 2024-03-16T10:07:49.187 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package unzip. 2024-03-16T10:07:49.192 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-16T10:07:49.213 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-16T10:07:49.219 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-16T10:07:49.253 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-16T10:07:49.259 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-16T10:07:49.529 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libradosstriper1. 2024-03-16T10:07:49.555 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:49.587 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libradosstriper1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:49.657 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package luarocks. 2024-03-16T10:07:49.683 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-16T10:07:49.715 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-16T10:07:50.015 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-common. 2024-03-16T10:07:50.042 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:50.073 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package librgw2. 2024-03-16T10:07:50.081 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:50.099 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../027-librgw2_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:50.139 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking librgw2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:50.666 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-rgw. 2024-03-16T10:07:50.693 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:50.733 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-rgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:51.069 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-16T10:07:51.096 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-16T10:07:51.127 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-16T10:07:51.430 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libradosstriper1. 2024-03-16T10:07:51.457 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:51.470 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-base. 2024-03-16T10:07:51.497 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libradosstriper1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:51.497 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:51.546 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-base (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:52.092 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph-common. 2024-03-16T10:07:52.116 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-16T10:07:52.119 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:52.142 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-16T10:07:52.158 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:52.182 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-16T10:07:52.468 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cheroot. 2024-03-16T10:07:52.495 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-16T10:07:52.543 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-16T10:07:52.854 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-16T10:07:52.870 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-16T10:07:52.904 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-16T10:07:53.174 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-16T10:07:53.200 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-16T10:07:53.240 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-16T10:07:53.455 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph-base. 2024-03-16T10:07:53.482 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:07:53.531 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph-base (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:07:53.542 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-16T10:07:53.569 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-16T10:07:53.609 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-16T10:07:53.921 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-tempora. 2024-03-16T10:07:53.947 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-16T10:07:53.987 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-16T10:07:54.068 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-16T10:07:54.094 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-16T10:07:54.126 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-16T10:07:54.256 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-portend. 2024-03-16T10:07:54.283 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-16T10:07:54.315 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-16T10:07:54.428 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-cheroot. 2024-03-16T10:07:54.456 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-16T10:07:54.495 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-16T10:07:54.592 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-16T10:07:54.619 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-16T10:07:54.659 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-16T10:07:54.764 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-16T10:07:54.780 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-16T10:07:54.814 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-16T10:07:54.979 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-16T10:07:55.006 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-16T10:07:55.045 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-16T10:07:55.117 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-16T10:07:55.144 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-16T10:07:55.184 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-16T10:07:55.331 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-natsort. 2024-03-16T10:07:55.358 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-16T10:07:55.398 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-16T10:07:55.486 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-16T10:07:55.513 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-16T10:07:55.553 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-16T10:07:55.709 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-logutils. 2024-03-16T10:07:55.736 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-16T10:07:55.776 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-16T10:07:55.856 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-tempora. 2024-03-16T10:07:55.882 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-16T10:07:55.922 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-16T10:07:56.225 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-portend. 2024-03-16T10:07:56.252 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-16T10:07:56.291 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-16T10:07:56.323 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-16T10:07:56.350 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-16T10:07:56.389 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-16T10:07:56.611 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-16T10:07:56.638 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-16T10:07:56.642 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-mako. 2024-03-16T10:07:56.658 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-16T10:07:56.678 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-16T10:07:56.700 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-16T10:07:57.020 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-16T10:07:57.047 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-16T10:07:57.086 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-16T10:07:57.139 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-16T10:07:57.165 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-16T10:07:57.205 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-16T10:07:57.373 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-16T10:07:57.400 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-16T10:07:57.439 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-16T10:07:57.559 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-natsort. 2024-03-16T10:07:57.585 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-16T10:07:57.634 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-16T10:07:57.742 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-webob. 2024-03-16T10:07:57.769 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-16T10:07:57.808 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-16T10:07:57.945 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-logutils. 2024-03-16T10:07:57.971 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-16T10:07:58.011 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-16T10:07:58.211 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-waitress. 2024-03-16T10:07:58.238 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-16T10:07:58.280 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-16T10:07:58.314 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-16T10:07:58.341 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-16T10:07:58.381 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-16T10:07:58.556 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-tempita. 2024-03-16T10:07:58.572 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-16T10:07:58.606 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-16T10:07:58.633 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-mako. 2024-03-16T10:07:58.649 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-16T10:07:58.683 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-16T10:07:58.875 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-paste. 2024-03-16T10:07:58.902 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-16T10:07:58.966 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-16T10:07:58.995 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-16T10:07:59.021 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-16T10:07:59.061 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-16T10:07:59.278 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-16T10:07:59.294 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-16T10:07:59.336 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-16T10:07:59.372 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-16T10:07:59.399 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-16T10:07:59.439 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-16T10:07:59.647 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-16T10:07:59.675 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-16T10:07:59.714 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-16T10:07:59.742 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-webob. 2024-03-16T10:07:59.768 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-16T10:07:59.808 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-16T10:07:59.975 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-webtest. 2024-03-16T10:07:59.999 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-16T10:08:00.033 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-16T10:08:00.170 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-waitress. 2024-03-16T10:08:00.196 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-16T10:08:00.240 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-16T10:08:00.336 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pecan. 2024-03-16T10:08:00.363 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-16T10:08:00.402 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-16T10:08:00.514 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-tempita. 2024-03-16T10:08:00.529 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-16T10:08:00.564 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-16T10:08:00.774 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-certifi. 2024-03-16T10:08:00.801 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-16T10:08:00.840 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-16T10:08:00.858 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-paste. 2024-03-16T10:08:00.885 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-16T10:08:01.267 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-16T10:08:01.928 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-16T10:08:01.944 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-16T10:08:01.987 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-16T10:08:02.128 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-idna. 2024-03-16T10:08:02.155 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-16T10:08:02.194 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-16T10:08:02.289 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-16T10:08:02.316 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-16T10:08:02.356 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-16T10:08:02.480 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-urllib3. 2024-03-16T10:08:02.507 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-16T10:08:02.547 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-16T10:08:02.633 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-webtest. 2024-03-16T10:08:02.649 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-16T10:08:02.683 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-16T10:08:02.841 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-requests. 2024-03-16T10:08:02.868 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-16T10:08:02.900 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-16T10:08:02.945 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-pecan. 2024-03-16T10:08:02.971 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-16T10:08:03.003 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-16T10:08:03.202 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-16T10:08:03.229 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-16T10:08:03.269 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-16T10:08:03.307 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-certifi. 2024-03-16T10:08:03.322 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-16T10:08:03.357 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-16T10:08:03.555 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-16T10:08:03.571 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:03.613 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:03.703 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-idna. 2024-03-16T10:08:03.729 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-16T10:08:03.777 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-16T10:08:04.043 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-16T10:08:04.070 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:04.096 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-urllib3. 2024-03-16T10:08:04.109 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:04.123 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-16T10:08:04.162 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-16T10:08:04.404 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-16T10:08:04.420 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-16T10:08:04.462 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-16T10:08:04.491 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-requests. 2024-03-16T10:08:04.517 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-16T10:08:04.557 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-16T10:08:04.715 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-openssl. 2024-03-16T10:08:04.742 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-16T10:08:04.781 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-16T10:08:04.860 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-16T10:08:04.887 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-16T10:08:04.927 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-16T10:08:05.134 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr. 2024-03-16T10:08:05.162 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:05.201 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:05.221 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-16T10:08:05.248 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:05.288 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:05.596 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mon. 2024-03-16T10:08:05.623 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:05.659 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-16T10:08:05.662 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mon (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:05.686 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:05.726 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:06.045 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-16T10:08:06.072 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-16T10:08:06.112 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-16T10:08:06.199 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-osd. 2024-03-16T10:08:06.226 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:06.257 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-osd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:06.415 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-openssl. 2024-03-16T10:08:06.441 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-16T10:08:06.481 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-16T10:08:06.759 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph-mgr. 2024-03-16T10:08:06.785 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:06.817 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph-mgr (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:07.352 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph. 2024-03-16T10:08:07.368 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../068-ceph_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:07.410 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:07.420 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph-mon. 2024-03-16T10:08:07.447 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:07.487 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph-mon (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:07.729 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-fuse. 2024-03-16T10:08:07.756 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:07.804 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:08.082 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph-osd. 2024-03-16T10:08:08.108 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:08.148 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph-osd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:08.174 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mds. 2024-03-16T10:08:08.201 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:08.240 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mds (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:08.635 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package cephadm. 2024-03-16T10:08:08.651 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../071-cephadm_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:08.684 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:08.971 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-16T10:08:08.998 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-16T10:08:09.037 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-16T10:08:09.159 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph. 2024-03-16T10:08:09.186 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../068-ceph_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:09.218 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:09.348 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-16T10:08:09.364 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-16T10:08:09.398 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-16T10:08:09.520 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph-fuse. 2024-03-16T10:08:09.547 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:09.587 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:09.973 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph-mds. 2024-03-16T10:08:10.000 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:10.039 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph-mds (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:10.436 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-babel. 2024-03-16T10:08:10.442 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package cephadm. 2024-03-16T10:08:10.465 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-16T10:08:10.469 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../071-cephadm_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:10.502 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-16T10:08:10.509 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:10.762 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-16T10:08:10.777 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-16T10:08:10.811 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-16T10:08:10.847 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jinja2. 2024-03-16T10:08:10.875 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-16T10:08:10.914 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-16T10:08:11.164 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-16T10:08:11.180 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-16T10:08:11.214 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-16T10:08:11.309 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-16T10:08:11.336 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:11.375 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:11.728 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-16T10:08:11.755 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-16T10:08:11.794 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-16T10:08:12.237 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-babel. 2024-03-16T10:08:12.256 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-routes. 2024-03-16T10:08:12.264 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-16T10:08:12.284 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-16T10:08:12.304 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-16T10:08:12.314 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-16T10:08:12.600 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-16T10:08:12.615 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-jinja2. 2024-03-16T10:08:12.616 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:12.641 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-16T10:08:12.650 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:12.681 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-16T10:08:13.043 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-16T10:08:13.069 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:13.109 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:13.111 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-16T10:08:13.140 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-16T10:08:13.178 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-16T10:08:13.462 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-16T10:08:13.489 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-16T10:08:13.529 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-16T10:08:13.699 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-joblib. 2024-03-16T10:08:13.726 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-16T10:08:13.765 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-16T10:08:13.807 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-routes. 2024-03-16T10:08:13.833 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-16T10:08:13.865 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-16T10:08:14.135 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-16T10:08:14.163 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-16T10:08:14.193 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-16T10:08:14.220 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:14.259 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:14.295 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-16T10:08:14.598 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-sklearn. 2024-03-16T10:08:14.626 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-16T10:08:14.664 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-16T10:08:14.729 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-16T10:08:14.745 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-16T10:08:14.787 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-16T10:08:15.299 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-joblib. 2024-03-16T10:08:15.326 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-16T10:08:15.365 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-16T10:08:15.448 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-16T10:08:15.476 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:15.514 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:15.669 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-16T10:08:15.696 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-16T10:08:15.812 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-16T10:08:16.124 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-sklearn. 2024-03-16T10:08:16.151 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-16T10:08:16.190 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-16T10:08:16.509 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cachetools. 2024-03-16T10:08:16.538 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-16T10:08:16.584 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-16T10:08:16.857 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-16T10:08:16.885 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:16.887 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-16T10:08:16.915 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-16T10:08:16.923 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:16.953 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-16T10:08:17.290 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-16T10:08:17.318 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-16T10:08:17.357 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-16T10:08:17.693 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rsa. 2024-03-16T10:08:17.721 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-16T10:08:17.760 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-16T10:08:18.113 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-google-auth. 2024-03-16T10:08:18.141 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-16T10:08:18.160 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-cachetools. 2024-03-16T10:08:18.187 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-16T10:08:18.188 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-16T10:08:18.226 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-16T10:08:18.504 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-16T10:08:18.531 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-16T10:08:18.571 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-16T10:08:18.575 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-16T10:08:18.602 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-16T10:08:18.649 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-16T10:08:18.858 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-16T10:08:18.884 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-16T10:08:18.915 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-16T10:08:18.994 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-websocket. 2024-03-16T10:08:19.022 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-16T10:08:19.061 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-16T10:08:19.160 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-rsa. 2024-03-16T10:08:19.175 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-16T10:08:19.210 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-16T10:08:19.381 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-16T10:08:19.408 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-16T10:08:19.472 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-16T10:08:19.539 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-google-auth. 2024-03-16T10:08:19.565 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-16T10:08:19.605 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-16T10:08:19.925 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-16T10:08:19.952 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-16T10:08:19.991 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-16T10:08:20.144 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-16T10:08:20.172 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:20.210 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:20.302 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-websocket. 2024-03-16T10:08:20.329 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-16T10:08:20.369 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-16T10:08:20.522 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-16T10:08:20.549 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-16T10:08:20.588 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-16T10:08:20.689 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-16T10:08:20.717 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-16T10:08:20.755 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-16T10:08:20.941 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-16T10:08:20.969 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-16T10:08:21.007 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-16T10:08:21.235 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package jq. 2024-03-16T10:08:21.260 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-16T10:08:21.293 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-16T10:08:21.444 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-16T10:08:21.460 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:21.502 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:21.546 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package socat. 2024-03-16T10:08:21.574 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-16T10:08:21.612 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-16T10:08:21.788 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-16T10:08:21.816 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-16T10:08:21.846 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-16T10:08:21.949 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package xmlstarlet. 2024-03-16T10:08:21.965 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-16T10:08:22.009 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-16T10:08:22.183 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-16T10:08:22.210 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-16T10:08:22.249 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-16T10:08:22.386 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-test. 2024-03-16T10:08:22.414 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:22.452 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-test (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:22.668 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package jq. 2024-03-16T10:08:22.684 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-16T10:08:22.743 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-16T10:08:23.021 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package socat. 2024-03-16T10:08:23.037 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-16T10:08:23.079 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-16T10:08:23.366 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package xmlstarlet. 2024-03-16T10:08:23.393 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-16T10:08:23.432 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-16T10:08:23.786 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph-test. 2024-03-16T10:08:23.802 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:23.844 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph-test (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:24.907 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-volume. 2024-03-16T10:08:24.935 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:24.973 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-volume (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:25.352 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-16T10:08:25.380 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:25.418 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking cephfs-mirror (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:25.738 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-colorama. 2024-03-16T10:08:25.766 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-16T10:08:25.804 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-16T10:08:26.040 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ceph-volume. 2024-03-16T10:08:26.065 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package cephfs-shell. 2024-03-16T10:08:26.068 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:26.082 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:26.098 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ceph-volume (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:26.115 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking cephfs-shell (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:26.351 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package cephfs-top. 2024-03-16T10:08:26.369 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:26.409 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking cephfs-top (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:26.427 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-16T10:08:26.454 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:26.493 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking cephfs-mirror (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:26.662 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-16T10:08:26.679 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:26.720 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libcephfs-dev (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:26.796 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-colorama. 2024-03-16T10:08:26.824 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-16T10:08:26.863 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-16T10:08:27.022 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-16T10:08:27.050 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-16T10:08:27.089 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-16T10:08:27.107 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package cephfs-shell. 2024-03-16T10:08:27.123 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:27.165 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking cephfs-shell (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:27.435 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package cephfs-top. 2024-03-16T10:08:27.435 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-16T10:08:27.462 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-16T10:08:27.463 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-16T10:08:27.501 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking cephfs-top (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:27.501 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-16T10:08:27.753 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-16T10:08:27.781 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:27.820 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libcephfs-dev (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:27.922 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package nvme-cli. 2024-03-16T10:08:27.950 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-16T10:08:27.988 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-16T10:08:28.289 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-16T10:08:28.317 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-16T10:08:28.347 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-16T10:08:28.410 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package pkg-config. 2024-03-16T10:08:28.438 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-16T10:08:28.476 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-16T10:08:28.643 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-16T10:08:28.671 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-16T10:08:28.696 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-16T10:08:28.701 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-16T10:08:28.712 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-16T10:08:28.746 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-16T10:08:29.097 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package nvme-cli. 2024-03-16T10:08:29.124 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-16T10:08:29.133 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-16T10:08:29.160 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-16T10:08:29.163 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-16T10:08:29.190 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-16T10:08:29.461 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pastescript. 2024-03-16T10:08:29.488 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-16T10:08:29.526 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-16T10:08:29.593 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package pkg-config. 2024-03-16T10:08:29.621 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-16T10:08:29.659 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-16T10:08:29.788 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pluggy. 2024-03-16T10:08:29.816 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-16T10:08:29.854 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-16T10:08:29.930 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-16T10:08:29.957 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-16T10:08:29.996 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-16T10:08:30.174 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-psutil. 2024-03-16T10:08:30.202 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-16T10:08:30.240 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-16T10:08:30.432 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-16T10:08:30.460 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-16T10:08:30.498 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-16T10:08:30.552 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-py. 2024-03-16T10:08:30.580 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-16T10:08:30.618 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-16T10:08:30.776 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-pastescript. 2024-03-16T10:08:30.792 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-16T10:08:30.826 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-16T10:08:30.938 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pygments. 2024-03-16T10:08:30.966 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-16T10:08:31.004 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-16T10:08:31.104 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-pluggy. 2024-03-16T10:08:31.132 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-16T10:08:31.171 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-16T10:08:31.474 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-psutil. 2024-03-16T10:08:31.492 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-16T10:08:31.502 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-16T10:08:31.520 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-16T10:08:31.540 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-16T10:08:31.558 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-16T10:08:31.862 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-toml. 2024-03-16T10:08:31.869 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-py. 2024-03-16T10:08:31.889 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-16T10:08:31.896 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-16T10:08:31.927 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-16T10:08:31.942 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-16T10:08:32.180 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pytest. 2024-03-16T10:08:32.208 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-16T10:08:32.221 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-pygments. 2024-03-16T10:08:32.246 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-16T10:08:32.249 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-16T10:08:32.287 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-16T10:08:32.583 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-simplejson. 2024-03-16T10:08:32.613 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-16T10:08:32.650 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-16T10:08:32.775 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-16T10:08:32.802 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-16T10:08:32.841 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-16T10:08:32.919 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-16T10:08:32.947 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-16T10:08:33.019 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-16T10:08:33.160 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-toml. 2024-03-16T10:08:33.188 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-16T10:08:33.227 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-16T10:08:33.496 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-pytest. 2024-03-16T10:08:33.524 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-16T10:08:33.554 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-16T10:08:33.881 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package radosgw. 2024-03-16T10:08:33.891 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package python3-simplejson. 2024-03-16T10:08:33.910 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../122-radosgw_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:33.918 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-16T10:08:33.947 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking radosgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:33.957 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-16T10:08:34.218 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-16T10:08:34.246 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-16T10:08:34.285 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-16T10:08:34.795 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package rbd-fuse. 2024-03-16T10:08:34.811 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:34.860 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking rbd-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:34.981 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package radosgw. 2024-03-16T10:08:35.009 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../122-radosgw_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:35.047 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking radosgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:35.199 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-16T10:08:35.457 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-16T10:08:35.591 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-16T10:08:35.833 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package rbd-fuse. 2024-03-16T10:08:35.860 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-16T10:08:35.861 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-16T10:08:35.899 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking rbd-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:36.185 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-16T10:08:36.237 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-16T10:08:36.496 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-16T10:08:36.622 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-16T10:08:36.627 INFO:teuthology.orchestra.run.smithi017.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-16T10:08:36.856 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-16T10:08:36.869 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:37.186 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-16T10:08:37.344 INFO:teuthology.orchestra.run.smithi017.stdout:Adding system user cephadm....done 2024-03-16T10:08:37.483 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-16T10:08:37.622 INFO:teuthology.orchestra.run.smithi174.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-16T10:08:37.690 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:37.759 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-16T10:08:38.019 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-16T10:08:38.144 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-16T10:08:38.207 INFO:teuthology.orchestra.run.smithi174.stdout:Adding system user cephadm....done 2024-03-16T10:08:38.412 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-16T10:08:38.613 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-16T10:08:38.655 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-16T10:08:38.781 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-16T10:08:38.881 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-16T10:08:39.091 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-16T10:08:39.115 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-16T10:08:39.241 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-16T10:08:39.476 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-16T10:08:39.484 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-16T10:08:39.736 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-16T10:08:39.760 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-16T10:08:39.878 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-16T10:08:39.987 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-16T10:08:40.188 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-16T10:08:40.255 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-16T10:08:40.506 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-16T10:08:40.548 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-16T10:08:40.757 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-16T10:08:40.799 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-16T10:08:41.025 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-16T10:08:41.050 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-16T10:08:41.284 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-ceph-argparse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:41.301 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-16T10:08:41.553 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-16T10:08:41.569 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-16T10:08:41.686 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-16T10:08:41.787 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-16T10:08:41.812 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-16T10:08:42.038 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-16T10:08:42.119 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-16T10:08:42.156 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-16T10:08:42.432 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-16T10:08:42.465 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-ceph-argparse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:42.716 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-16T10:08:42.787 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-16T10:08:42.822 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-16T10:08:42.858 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-16T10:08:42.885 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-16T10:08:42.968 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-16T10:08:43.011 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-16T10:08:43.210 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-16T10:08:43.336 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-16T10:08:43.487 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-16T10:08:43.588 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-16T10:08:43.604 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-16T10:08:43.889 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-16T10:08:43.988 INFO:teuthology.orchestra.run.smithi174.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-16T10:08:44.019 INFO:teuthology.orchestra.run.smithi174.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-16T10:08:44.074 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-16T10:08:44.173 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-16T10:08:44.200 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-16T10:08:44.483 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up zip (3.0-12build2) ... 2024-03-16T10:08:44.559 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-16T10:08:44.609 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-16T10:08:44.685 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-16T10:08:44.953 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-16T10:08:45.212 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-16T10:08:45.294 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-16T10:08:45.505 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up zip (3.0-12build2) ... 2024-03-16T10:08:45.554 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-16T10:08:45.606 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-16T10:08:45.671 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-16T10:08:45.772 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-16T10:08:46.073 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-16T10:08:46.233 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-16T10:08:46.441 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-16T10:08:46.492 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-16T10:08:46.618 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-16T10:08:46.692 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-16T10:08:46.769 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-16T10:08:47.077 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-16T10:08:47.087 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-16T10:08:47.370 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-16T10:08:47.480 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-16T10:08:47.688 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-16T10:08:47.706 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-16T10:08:48.039 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-16T10:08:48.074 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-16T10:08:48.331 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-16T10:08:48.367 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-16T10:08:48.432 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:48.668 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-16T10:08:48.726 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-idna (3.3-1) ... 2024-03-16T10:08:49.044 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-16T10:08:49.102 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-16T10:08:49.362 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-16T10:08:49.480 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:08:49.764 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-idna (3.3-1) ... 2024-03-16T10:08:50.048 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-16T10:08:50.257 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-16T10:08:50.484 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T10:08:50.618 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-16T10:08:50.894 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-16T10:08:51.020 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-16T10:08:51.146 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-16T10:08:51.237 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-16T10:08:51.464 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-16T10:08:51.489 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T10:08:51.598 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-16T10:08:51.732 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-16T10:08:51.841 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-16T10:08:51.958 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T10:08:51.967 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-16T10:08:52.075 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-16T10:08:52.093 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-16T10:08:52.385 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-16T10:08:52.393 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-16T10:08:52.653 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-16T10:08:52.653 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-16T10:08:52.888 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T10:08:52.929 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-16T10:08:52.997 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-16T10:08:53.188 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-16T10:08:53.282 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-16T10:08:53.447 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-16T10:08:53.533 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-16T10:08:53.724 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-16T10:08:53.792 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-16T10:08:53.824 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-16T10:08:54.077 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-16T10:08:54.109 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-16T10:08:54.235 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-16T10:08:54.344 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-16T10:08:54.528 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-16T10:08:54.612 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-16T10:08:54.730 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-16T10:08:54.837 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-16T10:08:54.989 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-16T10:08:55.097 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-16T10:08:55.107 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-16T10:08:55.222 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-16T10:08:55.348 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-16T10:08:55.399 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-16T10:08:55.420 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-16T10:08:55.675 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-16T10:08:55.701 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-16T10:08:55.801 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-16T10:08:55.952 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-16T10:08:56.077 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-16T10:08:56.194 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-16T10:08:56.203 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-16T10:08:56.287 INFO:teuthology.orchestra.run.smithi174.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-16T10:08:56.479 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-16T10:08:56.557 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-16T10:08:56.604 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-16T10:08:56.691 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-16T10:08:56.872 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T10:08:56.973 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-16T10:08:57.093 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-16T10:08:57.282 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-16T10:08:57.360 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-16T10:08:57.503 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-16T10:08:57.567 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-16T10:08:57.779 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T10:08:57.843 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-16T10:08:57.888 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-16T10:08:58.096 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-16T10:08:58.198 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-16T10:08:58.364 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-16T10:08:58.474 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-16T10:08:58.632 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-16T10:08:58.758 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-16T10:08:58.916 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-16T10:08:58.978 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-16T10:08:59.277 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-16T10:08:59.394 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-16T10:08:59.570 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-16T10:08:59.697 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-16T10:08:59.914 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-16T10:08:59.965 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-16T10:09:00.023 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librados2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:00.124 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:00.234 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-16T10:09:00.333 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-16T10:09:00.595 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libcephfs2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:00.618 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-16T10:09:00.720 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libradosstriper1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:00.846 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-16T10:09:00.962 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-16T10:09:01.104 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up librados2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:01.173 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librbd1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:01.214 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:01.274 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-modules-core (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:01.323 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-16T10:09:01.375 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:01.603 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-16T10:09:01.603 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-16T10:09:01.659 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libcephfs2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:01.785 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libradosstriper1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:01.911 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-16T10:09:02.076 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libcephfs-dev (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:02.187 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up librbd1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:02.202 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rados (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:02.289 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph-mgr-modules-core (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:02.328 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librgw2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:02.398 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:02.454 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up cephfs-top (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:02.580 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-16T10:09:02.608 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-16T10:09:02.608 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-16T10:09:03.124 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libcephfs-dev (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:03.250 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-rados (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:03.376 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up librgw2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:03.502 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up cephfs-top (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:03.628 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-16T10:09:03.875 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rbd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:04.001 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up rbd-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:04.127 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:04.253 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cephfs (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:04.379 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up cephfs-shell (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:04.530 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:04.923 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-rbd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:05.031 INFO:teuthology.orchestra.run.smithi017.stdout:Adding group ceph....done 2024-03-16T10:09:05.049 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up rbd-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:05.175 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-rgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:05.301 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up python3-cephfs (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:05.424 INFO:teuthology.orchestra.run.smithi017.stdout:Adding system user ceph....done 2024-03-16T10:09:05.427 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up cephfs-shell (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:05.553 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:05.584 INFO:teuthology.orchestra.run.smithi017.stdout:Setting system user ceph properties....done 2024-03-16T10:09:05.601 INFO:teuthology.orchestra.run.smithi017.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-16T10:09:05.748 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-16T10:09:06.071 INFO:teuthology.orchestra.run.smithi174.stdout:Adding group ceph....done 2024-03-16T10:09:06.105 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-16T10:09:06.413 INFO:teuthology.orchestra.run.smithi174.stdout:Adding system user ceph....done 2024-03-16T10:09:06.574 INFO:teuthology.orchestra.run.smithi174.stdout:Setting system user ceph properties....done 2024-03-16T10:09:06.593 INFO:teuthology.orchestra.run.smithi174.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-16T10:09:06.672 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up cephfs-mirror (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:06.732 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-16T10:09:06.893 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-16T10:09:06.893 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-16T10:09:07.039 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-16T10:09:07.357 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-test (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:07.457 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up radosgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:07.537 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up cephfs-mirror (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:07.728 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-16T10:09:07.729 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-16T10:09:07.938 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-16T10:09:07.938 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-16T10:09:08.224 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph-test (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:08.339 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up radosgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:08.426 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-base (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:08.807 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-16T10:09:08.814 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-16T10:09:08.814 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-16T10:09:09.308 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph-base (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:09.391 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mds (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:09.624 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-16T10:09:09.624 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-16T10:09:09.658 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-16T10:09:10.142 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:10.168 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph-mds (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:10.367 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-16T10:09:10.367 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-16T10:09:10.394 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-16T10:09:10.394 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-16T10:09:10.852 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-osd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:10.878 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph-mgr (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:11.103 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-16T10:09:11.104 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-16T10:09:11.137 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-16T10:09:11.137 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-16T10:09:11.595 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph-osd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:11.628 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:11.754 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:11.846 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-16T10:09:11.846 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-16T10:09:11.905 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mon (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:12.124 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-16T10:09:12.125 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-16T10:09:12.330 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:12.440 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:12.582 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph-mon (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:12.615 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:12.766 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:12.793 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-16T10:09:12.793 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-16T10:09:12.884 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-dashboard (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:13.035 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-volume (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:13.300 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph-mgr-cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-16T10:09:13.451 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:13.505 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-16T10:09:13.594 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph-mgr-dashboard (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:13.657 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-16T10:09:13.745 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ceph-volume (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-16T10:09:14.089 INFO:teuthology.orchestra.run.smithi174.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-16T10:09:14.182 INFO:teuthology.orchestra.run.smithi174.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-16T10:09:14.325 INFO:teuthology.orchestra.run.smithi174.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-16T10:09:14.716 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-16T10:09:15.369 INFO:teuthology.orchestra.run.smithi017.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-16T10:09:15.372 DEBUG:teuthology.orchestra.run.smithi017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-16T10:09:15.429 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-03-16T10:09:15.518 INFO:teuthology.orchestra.run.smithi174.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-16T10:09:15.610 INFO:teuthology.orchestra.run.smithi017.stdout:Building dependency tree... 2024-03-16T10:09:15.611 INFO:teuthology.orchestra.run.smithi017.stdout:Reading state information... 2024-03-16T10:09:15.776 INFO:teuthology.orchestra.run.smithi017.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-16T10:09:15.776 INFO:teuthology.orchestra.run.smithi017.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-16T10:09:15.776 INFO:teuthology.orchestra.run.smithi017.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-03-16T10:09:15.776 INFO:teuthology.orchestra.run.smithi017.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-03-16T10:09:15.776 INFO:teuthology.orchestra.run.smithi017.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-16T10:09:15.776 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T10:09:15.776 INFO:teuthology.orchestra.run.smithi017.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T10:09:15.776 INFO:teuthology.orchestra.run.smithi017.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T10:09:15.776 INFO:teuthology.orchestra.run.smithi017.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T10:09:15.776 INFO:teuthology.orchestra.run.smithi017.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-16T10:09:15.777 INFO:teuthology.orchestra.run.smithi017.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-16T10:09:15.777 INFO:teuthology.orchestra.run.smithi017.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T10:09:15.778 INFO:teuthology.orchestra.run.smithi017.stdout:The following additional packages will be installed: 2024-03-16T10:09:15.778 INFO:teuthology.orchestra.run.smithi017.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-16T10:09:15.778 INFO:teuthology.orchestra.run.smithi017.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-16T10:09:15.778 INFO:teuthology.orchestra.run.smithi017.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-03-16T10:09:15.778 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-16T10:09:15.778 INFO:teuthology.orchestra.run.smithi017.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-16T10:09:15.778 INFO:teuthology.orchestra.run.smithi017.stdout:Suggested packages: 2024-03-16T10:09:15.779 INFO:teuthology.orchestra.run.smithi017.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-16T10:09:15.779 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-03-16T10:09:15.779 INFO:teuthology.orchestra.run.smithi017.stdout: libengine-pkcs11-openssl 2024-03-16T10:09:15.816 INFO:teuthology.orchestra.run.smithi017.stdout:The following NEW packages will be installed: 2024-03-16T10:09:15.817 INFO:teuthology.orchestra.run.smithi017.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-16T10:09:15.817 INFO:teuthology.orchestra.run.smithi017.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-16T10:09:15.817 INFO:teuthology.orchestra.run.smithi017.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-16T10:09:15.817 INFO:teuthology.orchestra.run.smithi017.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-16T10:09:15.817 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-16T10:09:15.817 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-16T10:09:15.818 INFO:teuthology.orchestra.run.smithi017.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-16T10:09:15.818 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages will be upgraded: 2024-03-16T10:09:15.819 INFO:teuthology.orchestra.run.smithi017.stdout: libssl3 2024-03-16T10:09:15.915 INFO:teuthology.orchestra.run.smithi017.stdout:1 upgraded, 39 newly installed, 0 to remove and 283 not upgraded. 2024-03-16T10:09:15.915 INFO:teuthology.orchestra.run.smithi017.stdout:Need to get 54.6 MB of archives. 2024-03-16T10:09:15.915 INFO:teuthology.orchestra.run.smithi017.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-16T10:09:15.915 INFO:teuthology.orchestra.run.smithi017.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-16T10:09:16.038 INFO:teuthology.orchestra.run.smithi017.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-16T10:09:16.119 INFO:teuthology.orchestra.run.smithi017.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-16T10:09:16.135 INFO:teuthology.orchestra.run.smithi017.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-16T10:09:16.136 INFO:teuthology.orchestra.run.smithi017.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-16T10:09:16.139 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-16T10:09:16.141 INFO:teuthology.orchestra.run.smithi017.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-16T10:09:16.141 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-16T10:09:16.143 INFO:teuthology.orchestra.run.smithi017.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-16T10:09:16.145 INFO:teuthology.orchestra.run.smithi017.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-16T10:09:16.145 INFO:teuthology.orchestra.run.smithi017.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-16T10:09:16.146 INFO:teuthology.orchestra.run.smithi017.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-16T10:09:16.164 INFO:teuthology.orchestra.run.smithi017.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-16T10:09:16.185 INFO:teuthology.orchestra.run.smithi174.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-16T10:09:16.188 DEBUG:teuthology.orchestra.run.smithi174:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-16T10:09:16.283 INFO:teuthology.orchestra.run.smithi174.stdout:Reading package lists... 2024-03-16T10:09:16.478 INFO:teuthology.orchestra.run.smithi174.stdout:Building dependency tree... 2024-03-16T10:09:16.479 INFO:teuthology.orchestra.run.smithi174.stdout:Reading state information... 2024-03-16T10:09:16.643 INFO:teuthology.orchestra.run.smithi174.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-16T10:09:16.643 INFO:teuthology.orchestra.run.smithi174.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-16T10:09:16.643 INFO:teuthology.orchestra.run.smithi174.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-03-16T10:09:16.643 INFO:teuthology.orchestra.run.smithi174.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-03-16T10:09:16.643 INFO:teuthology.orchestra.run.smithi174.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-16T10:09:16.643 INFO:teuthology.orchestra.run.smithi174.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T10:09:16.643 INFO:teuthology.orchestra.run.smithi174.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T10:09:16.644 INFO:teuthology.orchestra.run.smithi174.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T10:09:16.644 INFO:teuthology.orchestra.run.smithi174.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T10:09:16.644 INFO:teuthology.orchestra.run.smithi174.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-16T10:09:16.644 INFO:teuthology.orchestra.run.smithi174.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-16T10:09:16.645 INFO:teuthology.orchestra.run.smithi174.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T10:09:16.645 INFO:teuthology.orchestra.run.smithi174.stdout:The following additional packages will be installed: 2024-03-16T10:09:16.645 INFO:teuthology.orchestra.run.smithi174.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-16T10:09:16.645 INFO:teuthology.orchestra.run.smithi174.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-16T10:09:16.645 INFO:teuthology.orchestra.run.smithi174.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-03-16T10:09:16.646 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-16T10:09:16.646 INFO:teuthology.orchestra.run.smithi174.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-16T10:09:16.646 INFO:teuthology.orchestra.run.smithi174.stdout:Suggested packages: 2024-03-16T10:09:16.647 INFO:teuthology.orchestra.run.smithi174.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-16T10:09:16.647 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-03-16T10:09:16.647 INFO:teuthology.orchestra.run.smithi174.stdout: libengine-pkcs11-openssl 2024-03-16T10:09:16.683 INFO:teuthology.orchestra.run.smithi174.stdout:The following NEW packages will be installed: 2024-03-16T10:09:16.683 INFO:teuthology.orchestra.run.smithi174.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-16T10:09:16.683 INFO:teuthology.orchestra.run.smithi174.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-16T10:09:16.684 INFO:teuthology.orchestra.run.smithi174.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-16T10:09:16.684 INFO:teuthology.orchestra.run.smithi174.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-16T10:09:16.684 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-16T10:09:16.684 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-16T10:09:16.684 INFO:teuthology.orchestra.run.smithi174.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-16T10:09:16.685 INFO:teuthology.orchestra.run.smithi174.stdout:The following packages will be upgraded: 2024-03-16T10:09:16.685 INFO:teuthology.orchestra.run.smithi174.stdout: libssl3 2024-03-16T10:09:16.742 INFO:teuthology.orchestra.run.smithi174.stdout:1 upgraded, 39 newly installed, 0 to remove and 283 not upgraded. 2024-03-16T10:09:16.742 INFO:teuthology.orchestra.run.smithi174.stdout:Need to get 54.6 MB of archives. 2024-03-16T10:09:16.742 INFO:teuthology.orchestra.run.smithi174.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-16T10:09:16.742 INFO:teuthology.orchestra.run.smithi174.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-16T10:09:16.745 INFO:teuthology.orchestra.run.smithi017.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-16T10:09:16.748 INFO:teuthology.orchestra.run.smithi017.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-16T10:09:16.749 INFO:teuthology.orchestra.run.smithi017.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-16T10:09:16.754 INFO:teuthology.orchestra.run.smithi017.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-16T10:09:16.783 INFO:teuthology.orchestra.run.smithi017.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-16T10:09:16.793 INFO:teuthology.orchestra.run.smithi017.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-16T10:09:16.796 INFO:teuthology.orchestra.run.smithi017.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-16T10:09:16.811 INFO:teuthology.orchestra.run.smithi017.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-16T10:09:16.812 INFO:teuthology.orchestra.run.smithi017.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-16T10:09:16.850 INFO:teuthology.orchestra.run.smithi017.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-16T10:09:16.851 INFO:teuthology.orchestra.run.smithi017.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-16T10:09:16.862 INFO:teuthology.orchestra.run.smithi174.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-16T10:09:16.864 INFO:teuthology.orchestra.run.smithi017.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-16T10:09:16.885 INFO:teuthology.orchestra.run.smithi017.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-16T10:09:16.889 INFO:teuthology.orchestra.run.smithi017.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-16T10:09:16.889 INFO:teuthology.orchestra.run.smithi017.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-16T10:09:16.889 INFO:teuthology.orchestra.run.smithi017.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-16T10:09:16.890 INFO:teuthology.orchestra.run.smithi017.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-16T10:09:16.908 INFO:teuthology.orchestra.run.smithi017.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-16T10:09:16.914 INFO:teuthology.orchestra.run.smithi017.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-16T10:09:16.961 INFO:teuthology.orchestra.run.smithi174.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-16T10:09:17.062 INFO:teuthology.orchestra.run.smithi174.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-16T10:09:17.063 INFO:teuthology.orchestra.run.smithi174.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-16T10:09:17.067 INFO:teuthology.orchestra.run.smithi174.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-16T10:09:17.070 INFO:teuthology.orchestra.run.smithi174.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-16T10:09:17.071 INFO:teuthology.orchestra.run.smithi174.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-16T10:09:17.073 INFO:teuthology.orchestra.run.smithi174.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-16T10:09:17.075 INFO:teuthology.orchestra.run.smithi174.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-16T10:09:17.076 INFO:teuthology.orchestra.run.smithi174.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-16T10:09:17.076 INFO:teuthology.orchestra.run.smithi174.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-16T10:09:17.093 INFO:teuthology.orchestra.run.smithi174.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-16T10:09:17.433 INFO:teuthology.orchestra.run.smithi017.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-16T10:09:17.433 INFO:teuthology.orchestra.run.smithi017.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-16T10:09:17.433 INFO:teuthology.orchestra.run.smithi017.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-16T10:09:17.434 INFO:teuthology.orchestra.run.smithi017.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-16T10:09:17.467 INFO:teuthology.orchestra.run.smithi017.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-16T10:09:17.467 INFO:teuthology.orchestra.run.smithi017.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-16T10:09:17.489 INFO:teuthology.orchestra.run.smithi017.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-16T10:09:17.490 INFO:teuthology.orchestra.run.smithi017.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-16T10:09:17.763 INFO:teuthology.orchestra.run.smithi174.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-16T10:09:17.766 INFO:teuthology.orchestra.run.smithi174.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-16T10:09:17.766 INFO:teuthology.orchestra.run.smithi174.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-16T10:09:17.770 INFO:teuthology.orchestra.run.smithi174.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-16T10:09:17.770 INFO:teuthology.orchestra.run.smithi174.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-16T10:09:17.777 INFO:teuthology.orchestra.run.smithi174.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-16T10:09:17.779 INFO:teuthology.orchestra.run.smithi174.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-16T10:09:17.789 INFO:teuthology.orchestra.run.smithi174.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-16T10:09:17.790 INFO:teuthology.orchestra.run.smithi174.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-16T10:09:17.819 INFO:teuthology.orchestra.run.smithi174.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-16T10:09:17.819 INFO:teuthology.orchestra.run.smithi174.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-16T10:09:17.831 INFO:teuthology.orchestra.run.smithi174.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-16T10:09:17.854 INFO:teuthology.orchestra.run.smithi174.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-16T10:09:17.857 INFO:teuthology.orchestra.run.smithi174.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-16T10:09:17.857 INFO:teuthology.orchestra.run.smithi174.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-16T10:09:17.858 INFO:teuthology.orchestra.run.smithi174.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-16T10:09:17.859 INFO:teuthology.orchestra.run.smithi174.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-16T10:09:17.876 INFO:teuthology.orchestra.run.smithi174.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-16T10:09:17.877 INFO:teuthology.orchestra.run.smithi174.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-16T10:09:17.915 INFO:teuthology.orchestra.run.smithi017.stdout:Preconfiguring packages ... 2024-03-16T10:09:18.077 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 54.6 MB in 2s (32.8 MB/s) 2024-03-16T10:09:18.141 INFO:teuthology.orchestra.run.smithi174.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-16T10:09:18.141 INFO:teuthology.orchestra.run.smithi174.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-16T10:09:18.141 INFO:teuthology.orchestra.run.smithi174.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-16T10:09:18.142 INFO:teuthology.orchestra.run.smithi174.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-16T10:09:18.142 INFO:teuthology.orchestra.run.smithi174.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-16T10:09:18.142 INFO:teuthology.orchestra.run.smithi174.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-16T10:09:18.153 INFO:teuthology.orchestra.run.smithi174.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-16T10:09:18.154 INFO:teuthology.orchestra.run.smithi174.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-16T10:09:18.222 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package flex. 2024-03-16T10:09:18.328 INFO:teuthology.orchestra.run.smithi017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173660 files and directories currently installed.) 2024-03-16T10:09:18.334 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-16T10:09:18.378 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-16T10:09:18.555 INFO:teuthology.orchestra.run.smithi174.stdout:Preconfiguring packages ... 2024-03-16T10:09:18.716 INFO:teuthology.orchestra.run.smithi174.stdout:Fetched 54.6 MB in 1s (37.4 MB/s) 2024-03-16T10:09:18.841 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package flex. 2024-03-16T10:09:18.937 INFO:teuthology.orchestra.run.smithi174.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173660 files and directories currently installed.) 2024-03-16T10:09:18.943 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-16T10:09:18.953 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-16T10:09:19.014 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-16T10:09:19.134 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-16T10:09:19.553 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-16T10:09:19.572 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-16T10:09:19.744 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-16T10:09:20.102 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package bison. 2024-03-16T10:09:20.208 INFO:teuthology.orchestra.run.smithi017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173733 files and directories currently installed.) 2024-03-16T10:09:20.214 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-16T10:09:20.264 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-16T10:09:20.270 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-16T10:09:20.671 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-16T10:09:20.699 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-16T10:09:20.737 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-16T10:09:20.846 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package bison. 2024-03-16T10:09:20.951 INFO:teuthology.orchestra.run.smithi174.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173733 files and directories currently installed.) 2024-03-16T10:09:20.957 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-16T10:09:21.007 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-16T10:09:21.099 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package dump. 2024-03-16T10:09:21.127 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-16T10:09:21.165 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking dump (0.4b47-3) ... 2024-03-16T10:09:21.382 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-16T10:09:21.410 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-16T10:09:21.427 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package indent. 2024-03-16T10:09:21.443 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-16T10:09:21.449 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-16T10:09:21.477 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-16T10:09:21.738 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-16T10:09:21.760 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package dump. 2024-03-16T10:09:21.766 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-16T10:09:21.788 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-16T10:09:21.804 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-16T10:09:21.829 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking dump (0.4b47-3) ... 2024-03-16T10:09:22.088 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package indent. 2024-03-16T10:09:22.104 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-16T10:09:22.132 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-16T10:09:22.146 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-16T10:09:22.161 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-16T10:09:22.199 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-16T10:09:22.415 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-16T10:09:22.432 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-16T10:09:22.460 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-16T10:09:22.474 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-16T10:09:22.477 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-16T10:09:22.518 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-16T10:09:22.786 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-16T10:09:22.814 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-16T10:09:22.830 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-16T10:09:22.844 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-16T10:09:22.858 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-16T10:09:22.896 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-16T10:09:23.096 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-16T10:09:23.112 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-16T10:09:23.124 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-16T10:09:23.152 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-16T10:09:23.154 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-16T10:09:23.190 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-16T10:09:23.443 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libjson-perl. 2024-03-16T10:09:23.471 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-16T10:09:23.499 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-16T10:09:23.509 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-16T10:09:23.527 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-16T10:09:23.566 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-16T10:09:23.818 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-16T10:09:23.829 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-16T10:09:23.834 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-16T10:09:23.857 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-16T10:09:23.876 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-16T10:09:23.895 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-16T10:09:24.104 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libjson-perl. 2024-03-16T10:09:24.120 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-16T10:09:24.154 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-16T10:09:24.457 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-16T10:09:24.484 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-16T10:09:24.523 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-16T10:09:25.634 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-16T10:09:25.660 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-16T10:09:25.698 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T10:09:25.967 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-16T10:09:25.996 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-16T10:09:26.033 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T10:09:26.285 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-16T10:09:26.313 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-16T10:09:26.345 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-16T10:09:26.360 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T10:09:26.373 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-16T10:09:26.411 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T10:09:26.646 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-16T10:09:26.661 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-16T10:09:26.704 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T10:09:26.714 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-16T10:09:26.742 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-16T10:09:26.780 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-16T10:09:27.040 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-16T10:09:27.068 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-16T10:09:27.106 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T10:09:27.108 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-16T10:09:27.136 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-16T10:09:27.174 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-16T10:09:27.471 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-16T10:09:27.500 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-16T10:09:27.509 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-16T10:09:27.529 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T10:09:27.537 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-16T10:09:27.584 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-16T10:09:27.840 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-16T10:09:27.869 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-16T10:09:27.907 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T10:09:27.991 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-16T10:09:28.014 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-16T10:09:28.066 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-16T10:09:28.219 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-16T10:09:28.234 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-16T10:09:28.277 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T10:09:28.414 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-16T10:09:28.442 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-16T10:09:28.481 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T10:09:28.519 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-16T10:09:28.548 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-16T10:09:28.586 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-16T10:09:28.826 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-16T10:09:28.853 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-16T10:09:28.892 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T10:09:29.115 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-16T10:09:29.143 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-16T10:09:29.181 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-16T10:09:29.236 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-16T10:09:29.264 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-16T10:09:29.311 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T10:09:29.534 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package modemmanager. 2024-03-16T10:09:29.562 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-16T10:09:29.600 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T10:09:29.605 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-16T10:09:29.632 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-16T10:09:29.671 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-16T10:09:30.158 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-16T10:09:30.175 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-16T10:09:30.181 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package network-manager. 2024-03-16T10:09:30.208 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-16T10:09:30.210 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-16T10:09:30.247 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-16T10:09:30.888 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ppp. 2024-03-16T10:09:30.916 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-16T10:09:31.020 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package modemmanager. 2024-03-16T10:09:31.048 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-16T10:09:31.087 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T10:09:31.192 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-16T10:09:31.500 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package pptp-linux. 2024-03-16T10:09:31.529 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-16T10:09:31.567 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-16T10:09:31.634 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package network-manager. 2024-03-16T10:09:31.662 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-16T10:09:31.700 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-16T10:09:31.819 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-16T10:09:31.836 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-16T10:09:31.877 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-16T10:09:32.181 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-16T10:09:32.209 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-16T10:09:32.247 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking postgresql-client-common (238) ... 2024-03-16T10:09:32.341 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package ppp. 2024-03-16T10:09:32.369 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-16T10:09:32.534 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-16T10:09:32.562 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-16T10:09:32.601 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T10:09:32.659 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-16T10:09:32.953 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package pptp-linux. 2024-03-16T10:09:32.969 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-16T10:09:33.003 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-16T10:09:33.206 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package postgresql-common. 2024-03-16T10:09:33.222 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-16T10:09:33.234 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-16T10:09:33.250 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-16T10:09:33.291 INFO:teuthology.orchestra.run.smithi017.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-16T10:09:33.312 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking postgresql-common (238) ... 2024-03-16T10:09:33.328 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-16T10:09:33.634 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-16T10:09:33.661 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-16T10:09:33.700 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking postgresql-client-common (238) ... 2024-03-16T10:09:33.754 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package postgresql-14. 2024-03-16T10:09:33.783 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-16T10:09:33.823 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T10:09:34.004 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-16T10:09:34.031 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-16T10:09:34.070 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T10:09:34.650 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package postgresql-common. 2024-03-16T10:09:34.678 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-16T10:09:34.727 INFO:teuthology.orchestra.run.smithi174.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-16T10:09:34.756 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking postgresql-common (238) ... 2024-03-16T10:09:35.023 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package postgresql. 2024-03-16T10:09:35.040 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-16T10:09:35.073 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking postgresql (14+238) ... 2024-03-16T10:09:35.232 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package postgresql-14. 2024-03-16T10:09:35.260 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-16T10:09:35.302 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T10:09:35.316 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package postgresql-client. 2024-03-16T10:09:35.345 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-16T10:09:35.383 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking postgresql-client (14+238) ... 2024-03-16T10:09:35.627 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-16T10:09:35.644 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-16T10:09:35.685 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-16T10:09:36.013 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package tcl8.6. 2024-03-16T10:09:36.041 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-16T10:09:36.079 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-16T10:09:36.315 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package tcl. 2024-03-16T10:09:36.332 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-16T10:09:36.378 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-16T10:09:36.668 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package wpasupplicant. 2024-03-16T10:09:36.696 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-16T10:09:36.726 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-16T10:09:36.769 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package postgresql. 2024-03-16T10:09:36.786 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-16T10:09:36.827 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking postgresql (14+238) ... 2024-03-16T10:09:37.062 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package postgresql-client. 2024-03-16T10:09:37.072 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-16T10:09:37.091 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-16T10:09:37.100 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-16T10:09:37.129 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking postgresql-client (14+238) ... 2024-03-16T10:09:37.138 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-16T10:09:37.372 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-16T10:09:37.401 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-16T10:09:37.439 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-16T10:09:37.716 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-16T10:09:37.745 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-16T10:09:37.767 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package tcl8.6. 2024-03-16T10:09:37.791 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-16T10:09:37.795 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-16T10:09:37.833 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-16T10:09:38.080 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up postgresql-client-common (238) ... 2024-03-16T10:09:38.102 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package tcl. 2024-03-16T10:09:38.130 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-16T10:09:38.173 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-16T10:09:38.289 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-16T10:09:38.415 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-16T10:09:38.505 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package wpasupplicant. 2024-03-16T10:09:38.532 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T10:09:38.533 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-16T10:09:38.571 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-16T10:09:38.633 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-16T10:09:38.742 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-16T10:09:38.868 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T10:09:38.934 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-16T10:09:38.950 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-16T10:09:38.992 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-16T10:09:39.320 INFO:teuthology.orchestra.run.smithi174.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-16T10:09:39.348 INFO:teuthology.orchestra.run.smithi174.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-16T10:09:39.395 INFO:teuthology.orchestra.run.smithi174.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-16T10:09:39.407 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-16T10:09:39.499 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-16T10:09:39.682 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up postgresql-client-common (238) ... 2024-03-16T10:09:39.837 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-16T10:09:39.837 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-16T10:09:39.892 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-16T10:09:40.019 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-16T10:09:40.145 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T10:09:40.262 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-16T10:09:40.286 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-16T10:09:40.363 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-16T10:09:40.411 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-16T10:09:40.489 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T10:09:40.537 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-16T10:09:40.671 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-16T10:09:40.797 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-16T10:09:40.940 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-16T10:09:41.011 INFO:teuthology.orchestra.run.smithi174.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-16T10:09:41.057 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-16T10:09:41.096 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-16T10:09:41.458 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-16T10:09:41.458 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-16T10:09:41.662 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-16T10:09:41.788 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up dump (0.4b47-3) ... 2024-03-16T10:09:41.882 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-16T10:09:41.941 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-16T10:09:41.948 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T10:09:42.042 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-16T10:09:42.074 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-16T10:09:42.159 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-16T10:09:42.175 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-16T10:09:42.276 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-16T10:09:42.285 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-16T10:09:42.385 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-16T10:09:42.411 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-16T10:09:42.476 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-16T10:09:42.537 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-16T10:09:42.545 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-16T10:09:42.663 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-16T10:09:42.671 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-16T10:09:42.847 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-16T10:09:42.973 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T10:09:43.099 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-16T10:09:43.250 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up postgresql-client (14+238) ... 2024-03-16T10:09:43.260 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-16T10:09:43.351 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-16T10:09:43.385 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up dump (0.4b47-3) ... 2024-03-16T10:09:43.497 INFO:teuthology.orchestra.run.smithi174.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-16T10:09:43.554 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T10:09:43.680 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-16T10:09:43.831 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:09:43.831 INFO:teuthology.orchestra.run.smithi017.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-16T10:09:43.831 INFO:teuthology.orchestra.run.smithi017.stdout:which means they are currently configured by ifupdown: 2024-03-16T10:09:43.831 INFO:teuthology.orchestra.run.smithi017.stdout:- ens1f0 2024-03-16T10:09:43.831 INFO:teuthology.orchestra.run.smithi017.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-16T10:09:43.831 INFO:teuthology.orchestra.run.smithi017.stdout:remove their configuration from /etc/network/interfaces. 2024-03-16T10:09:43.831 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:09:43.869 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-16T10:09:43.953 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-16T10:09:44.058 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-16T10:09:44.182 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-16T10:09:44.256 INFO:teuthology.orchestra.run.smithi174.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-16T10:09:44.295 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-16T10:09:44.317 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-16T10:09:44.426 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-16T10:09:44.603 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-16T10:09:44.635 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-16T10:09:44.728 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T10:09:44.854 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-16T10:09:45.022 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up postgresql-client (14+238) ... 2024-03-16T10:09:45.095 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T10:09:45.148 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-16T10:09:45.255 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-16T10:09:45.365 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-16T10:09:45.507 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up postgresql-common (238) ... 2024-03-16T10:09:45.623 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:09:45.623 INFO:teuthology.orchestra.run.smithi174.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-16T10:09:45.623 INFO:teuthology.orchestra.run.smithi174.stdout:which means they are currently configured by ifupdown: 2024-03-16T10:09:45.623 INFO:teuthology.orchestra.run.smithi174.stdout:- enp3s0f1 2024-03-16T10:09:45.623 INFO:teuthology.orchestra.run.smithi174.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-16T10:09:45.623 INFO:teuthology.orchestra.run.smithi174.stdout:remove their configuration from /etc/network/interfaces. 2024-03-16T10:09:45.623 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:09:45.746 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-16T10:09:46.073 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-16T10:09:46.404 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-16T10:09:46.417 INFO:teuthology.orchestra.run.smithi017.stdout:Adding user postgres to group ssl-cert 2024-03-16T10:09:46.659 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:09:46.659 INFO:teuthology.orchestra.run.smithi017.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-16T10:09:46.864 INFO:teuthology.orchestra.run.smithi017.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-16T10:09:46.864 INFO:teuthology.orchestra.run.smithi017.stdout:Removing obsolete dictionary files: 2024-03-16T10:09:46.984 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T10:09:47.136 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-16T10:09:47.261 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-16T10:09:47.387 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up postgresql-common (238) ... 2024-03-16T10:09:47.412 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-16T10:09:47.817 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T10:09:48.164 INFO:teuthology.orchestra.run.smithi017.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-16T10:09:48.175 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-16T10:09:48.224 INFO:teuthology.orchestra.run.smithi017.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-16T10:09:48.224 INFO:teuthology.orchestra.run.smithi017.stdout:This user must also own the server process. 2024-03-16T10:09:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:09:48.224 INFO:teuthology.orchestra.run.smithi017.stdout:The database cluster will be initialized with locale "en_US". 2024-03-16T10:09:48.224 INFO:teuthology.orchestra.run.smithi017.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-16T10:09:48.224 INFO:teuthology.orchestra.run.smithi017.stdout:The default text search configuration will be set to "english". 2024-03-16T10:09:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:09:48.224 INFO:teuthology.orchestra.run.smithi017.stdout:Data page checksums are disabled. 2024-03-16T10:09:48.224 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:09:48.224 INFO:teuthology.orchestra.run.smithi017.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-16T10:09:48.225 INFO:teuthology.orchestra.run.smithi017.stdout:creating subdirectories ... ok 2024-03-16T10:09:48.225 INFO:teuthology.orchestra.run.smithi017.stdout:selecting dynamic shared memory implementation ... posix 2024-03-16T10:09:48.239 INFO:teuthology.orchestra.run.smithi017.stdout:selecting default max_connections ... 100 2024-03-16T10:09:48.261 INFO:teuthology.orchestra.run.smithi017.stdout:selecting default shared_buffers ... 128MB 2024-03-16T10:09:48.263 INFO:teuthology.orchestra.run.smithi174.stdout:Adding user postgres to group ssl-cert 2024-03-16T10:09:48.269 INFO:teuthology.orchestra.run.smithi017.stdout:selecting default time zone ... Etc/UTC 2024-03-16T10:09:48.270 INFO:teuthology.orchestra.run.smithi017.stdout:creating configuration files ... ok 2024-03-16T10:09:48.388 INFO:teuthology.orchestra.run.smithi017.stdout:running bootstrap script ... ok 2024-03-16T10:09:48.500 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:09:48.500 INFO:teuthology.orchestra.run.smithi174.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-16T10:09:48.676 INFO:teuthology.orchestra.run.smithi174.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-16T10:09:48.676 INFO:teuthology.orchestra.run.smithi174.stdout:Removing obsolete dictionary files: 2024-03-16T10:09:49.036 INFO:teuthology.orchestra.run.smithi017.stdout:performing post-bootstrap initialization ... ok 2024-03-16T10:09:49.226 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-16T10:09:49.613 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T10:09:49.698 INFO:teuthology.orchestra.run.smithi017.stdout:syncing data to disk ... ok 2024-03-16T10:09:49.938 INFO:teuthology.orchestra.run.smithi174.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-16T10:09:49.962 INFO:teuthology.orchestra.run.smithi174.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-16T10:09:50.001 INFO:teuthology.orchestra.run.smithi174.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-16T10:09:50.001 INFO:teuthology.orchestra.run.smithi174.stdout:This user must also own the server process. 2024-03-16T10:09:50.002 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:09:50.002 INFO:teuthology.orchestra.run.smithi174.stdout:The database cluster will be initialized with locale "en_US". 2024-03-16T10:09:50.002 INFO:teuthology.orchestra.run.smithi174.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-16T10:09:50.002 INFO:teuthology.orchestra.run.smithi174.stdout:The default text search configuration will be set to "english". 2024-03-16T10:09:50.002 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:09:50.002 INFO:teuthology.orchestra.run.smithi174.stdout:Data page checksums are disabled. 2024-03-16T10:09:50.002 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:09:50.002 INFO:teuthology.orchestra.run.smithi174.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-16T10:09:50.002 INFO:teuthology.orchestra.run.smithi174.stdout:creating subdirectories ... ok 2024-03-16T10:09:50.002 INFO:teuthology.orchestra.run.smithi174.stdout:selecting dynamic shared memory implementation ... posix 2024-03-16T10:09:50.017 INFO:teuthology.orchestra.run.smithi174.stdout:selecting default max_connections ... 100 2024-03-16T10:09:50.037 INFO:teuthology.orchestra.run.smithi174.stdout:selecting default shared_buffers ... 128MB 2024-03-16T10:09:50.046 INFO:teuthology.orchestra.run.smithi174.stdout:selecting default time zone ... Etc/UTC 2024-03-16T10:09:50.047 INFO:teuthology.orchestra.run.smithi174.stdout:creating configuration files ... ok 2024-03-16T10:09:50.106 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-16T10:09:50.160 INFO:teuthology.orchestra.run.smithi174.stdout:running bootstrap script ... ok 2024-03-16T10:09:50.873 INFO:teuthology.orchestra.run.smithi174.stdout:performing post-bootstrap initialization ... ok 2024-03-16T10:09:51.565 INFO:teuthology.orchestra.run.smithi174.stdout:syncing data to disk ... ok 2024-03-16T10:09:51.958 INFO:teuthology.orchestra.run.smithi174.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-16T10:09:53.073 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T10:09:53.174 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-16T10:09:53.300 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up postgresql (14+238) ... 2024-03-16T10:09:53.426 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T10:09:53.552 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T10:09:53.840 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-16T10:09:53.840 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-16T10:09:54.455 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-16T10:09:54.727 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T10:09:54.748 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-16T10:09:54.878 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-16T10:09:54.883 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-16T10:09:55.004 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up postgresql (14+238) ... 2024-03-16T10:09:55.122 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T10:09:55.222 INFO:teuthology.orchestra.run.smithi174.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T10:09:55.468 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-16T10:09:55.468 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-16T10:09:56.033 INFO:teuthology.orchestra.run.smithi174.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-16T10:09:56.269 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-16T10:09:56.344 INFO:teuthology.orchestra.run.smithi174.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-16T10:09:56.368 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-16T10:09:56.495 INFO:teuthology.orchestra.run.smithi174.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-16T10:09:56.867 INFO:teuthology.orchestra.run.smithi017.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-16T10:09:56.872 DEBUG:teuthology.parallel:result is None 2024-03-16T10:09:57.831 INFO:teuthology.orchestra.run.smithi174.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-16T10:09:57.930 INFO:teuthology.orchestra.run.smithi174.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-16T10:09:58.413 INFO:teuthology.orchestra.run.smithi174.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-16T10:09:58.418 DEBUG:teuthology.parallel:result is None 2024-03-16T10:09:58.418 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:09:58.555 DEBUG:teuthology.orchestra.run.smithi017:> dpkg-query -W -f '${Version}' ceph 2024-03-16T10:09:58.575 INFO:teuthology.orchestra.run.smithi017.stdout:19.0.0-2162-g3d78f5ba-1jammy 2024-03-16T10:09:58.576 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162-g3d78f5ba-1jammy 2024-03-16T10:09:58.576 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162-g3d78f5ba-1jammy is installed. 2024-03-16T10:09:58.577 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-16T10:09:58.709 DEBUG:teuthology.orchestra.run.smithi174:> dpkg-query -W -f '${Version}' ceph 2024-03-16T10:09:58.729 INFO:teuthology.orchestra.run.smithi174.stdout:19.0.0-2162-g3d78f5ba-1jammy 2024-03-16T10:09:58.729 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162-g3d78f5ba-1jammy 2024-03-16T10:09:58.729 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162-g3d78f5ba-1jammy is installed. 2024-03-16T10:09:58.733 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-16T10:09:58.733 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:09:58.733 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T10:09:58.748 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:09:58.748 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T10:09:58.784 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-16T10:09:58.784 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:09:58.784 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T10:09:58.802 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T10:09:58.861 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:09:58.861 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T10:09:58.874 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T10:09:58.930 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-16T10:09:58.930 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:09:58.930 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T10:09:58.943 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T10:09:58.997 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:09:58.997 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T10:09:59.010 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T10:09:59.065 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-16T10:09:59.065 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:09:59.065 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T10:09:59.079 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T10:09:59.134 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:09:59.134 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T10:09:59.147 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T10:09:59.202 INFO:teuthology.run_tasks:Running task ceph... 2024-03-16T10:09:59.283 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-16T10:09:59.283 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 777 /var/log/ceph 2024-03-16T10:09:59.286 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /var/log/ceph 2024-03-16T10:09:59.299 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-16T10:09:59.299 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-16T10:09:59.343 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-16T10:09:59.355 INFO:tasks.ceph:Creating extra log directories... 2024-03-16T10:09:59.356 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-16T10:09:59.399 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-16T10:09:59.417 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-16T10:09:59.417 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-16T10:09:59.417 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-16T10:09:59.417 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:09:59.417 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-16T10:09:59.458 DEBUG:teuthology.orchestra.run.smithi017:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-16T10:09:59.534 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:09:59.535 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-16T10:09:59.540 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-16T10:09:59.617 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-16T10:09:59.617 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-16T10:09:59.617 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360', 'branch': 'wip-batrick-testing-20240314.183243', 'description': 'fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7602360', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-batrick-testing-20240314.183243'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20240314.183243', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 4770, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'sleep_before_teardown': 0, 'subset': '498/512', 'suite': 'fs', 'suite_branch': 'wip-batrick-testing-20240314.183243', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi017.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAzG3WuJoO9CsYpabwEtXZIEsNWtuwVD9c72O5USbYQ8CXtNNbr/289XNGIWmEdcOxZjFtJOiM+NbxM2k1OpnIk=', 'smithi174.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKvDj+uaglxBy4cNsSC8W66C4LA0RrviET8P2Pr+TEqKog/O1npB2zywdCmM4QOM8TIu5s5VxLpqZoxG/2ZLGzg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_cephfs_shell']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-15_13:00:41', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-16T10:09:59.618 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-16T10:09:59.620 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-16T10:09:59.659 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m0777 -- /var/run/ceph 2024-03-16T10:09:59.667 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m0777 -- /var/run/ceph 2024-03-16T10:09:59.715 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:09:59.715 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T10:09:59.726 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T10:09:59.726 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_1 2024-03-16T10:09:59.774 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T10:09:59.774 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:09:59.774 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 872 Links: 1 2024-03-16T10:09:59.775 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:09:59.775 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-03-16 10:03:59.171194179 +0000 2024-03-16T10:09:59.775 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-03-16 10:03:58.939196241 +0000 2024-03-16T10:09:59.775 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-03-16 10:03:58.939196241 +0000 2024-03-16T10:09:59.775 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2024-03-16T10:09:59.775 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T10:09:59.829 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-03-16T10:09:59.829 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-03-16T10:09:59.829 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000257196 s, 2.0 MB/s 2024-03-16T10:09:59.831 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T10:09:59.880 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_2 2024-03-16T10:09:59.930 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T10:09:59.930 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:09:59.931 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 891 Links: 1 2024-03-16T10:09:59.931 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:09:59.931 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-03-16 10:03:59.655189876 +0000 2024-03-16T10:09:59.931 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-03-16 10:03:59.395192187 +0000 2024-03-16T10:09:59.931 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-03-16 10:03:59.395192187 +0000 2024-03-16T10:09:59.931 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2024-03-16T10:09:59.931 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T10:09:59.985 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-03-16T10:09:59.985 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-03-16T10:09:59.985 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000247199 s, 2.1 MB/s 2024-03-16T10:09:59.986 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T10:10:00.036 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_3 2024-03-16T10:10:00.082 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T10:10:00.082 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:10:00.082 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-03-16T10:10:00.082 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:10:00.083 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-03-16 10:04:00.175185254 +0000 2024-03-16T10:10:00.083 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-03-16 10:03:59.879187885 +0000 2024-03-16T10:10:00.083 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-03-16 10:03:59.879187885 +0000 2024-03-16T10:10:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2024-03-16T10:10:00.083 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T10:10:00.137 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-03-16T10:10:00.137 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-03-16T10:10:00.137 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000312525 s, 1.6 MB/s 2024-03-16T10:10:00.138 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T10:10:00.188 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_4 2024-03-16T10:10:00.234 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T10:10:00.234 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:10:00.235 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-03-16T10:10:00.235 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:10:00.235 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-03-16 10:04:00.647181058 +0000 2024-03-16T10:10:00.235 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-03-16 10:04:00.387183370 +0000 2024-03-16T10:10:00.235 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-03-16 10:04:00.387183370 +0000 2024-03-16T10:10:00.235 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2024-03-16T10:10:00.235 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T10:10:00.288 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-03-16T10:10:00.289 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-03-16T10:10:00.289 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000314095 s, 1.6 MB/s 2024-03-16T10:10:00.290 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T10:10:00.339 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T10:10:00.340 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:10:00.340 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T10:10:00.345 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T10:10:00.345 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2024-03-16T10:10:00.391 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T10:10:00.391 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:10:00.391 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 881 Links: 1 2024-03-16T10:10:00.391 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:10:00.391 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-03-16 10:04:00.348436618 +0000 2024-03-16T10:10:00.391 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-03-16 10:04:00.080439010 +0000 2024-03-16T10:10:00.392 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-03-16 10:04:00.080439010 +0000 2024-03-16T10:10:00.392 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2024-03-16T10:10:00.392 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T10:10:00.445 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-03-16T10:10:00.446 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-03-16T10:10:00.446 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000450146 s, 1.1 MB/s 2024-03-16T10:10:00.447 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T10:10:00.497 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2024-03-16T10:10:00.543 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T10:10:00.543 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:10:00.543 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 896 Links: 1 2024-03-16T10:10:00.543 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:10:00.543 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-03-16 10:04:00.864432016 +0000 2024-03-16T10:10:00.543 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-03-16 10:04:00.600434371 +0000 2024-03-16T10:10:00.544 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-03-16 10:04:00.600434371 +0000 2024-03-16T10:10:00.544 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2024-03-16T10:10:00.544 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T10:10:00.597 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-03-16T10:10:00.597 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-03-16T10:10:00.597 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000345597 s, 1.5 MB/s 2024-03-16T10:10:00.598 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T10:10:00.648 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2024-03-16T10:10:00.695 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T10:10:00.695 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:10:00.695 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 910 Links: 1 2024-03-16T10:10:00.695 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:10:00.695 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-03-16 10:04:01.384427377 +0000 2024-03-16T10:10:00.695 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-03-16 10:04:01.116429768 +0000 2024-03-16T10:10:00.695 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-03-16 10:04:01.116429768 +0000 2024-03-16T10:10:00.695 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2024-03-16T10:10:00.696 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T10:10:00.749 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-03-16T10:10:00.749 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-03-16T10:10:00.749 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000414006 s, 1.2 MB/s 2024-03-16T10:10:00.751 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T10:10:00.800 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2024-03-16T10:10:00.847 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T10:10:00.847 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T10:10:00.847 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 929 Links: 1 2024-03-16T10:10:00.847 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T10:10:00.847 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-03-16 10:04:01.880422953 +0000 2024-03-16T10:10:00.847 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-03-16 10:04:01.616425306 +0000 2024-03-16T10:10:00.847 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-03-16 10:04:01.616425306 +0000 2024-03-16T10:10:00.847 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2024-03-16T10:10:00.848 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T10:10:00.901 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-03-16T10:10:00.901 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-03-16T10:10:00.901 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000329477 s, 1.6 MB/s 2024-03-16T10:10:00.902 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T10:10:00.952 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T10:10:00.952 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi017.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi174.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-16T10:10:00.952 INFO:tasks.ceph:Generating config... 2024-03-16T10:10:00.954 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-16T10:10:00.954 INFO:tasks.ceph:[client] debug client = 20 2024-03-16T10:10:00.954 INFO:tasks.ceph:[client] debug ms = 1 2024-03-16T10:10:00.954 INFO:tasks.ceph:[client] pid file = 2024-03-16T10:10:00.954 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-16T10:10:00.954 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-16T10:10:00.954 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-16T10:10:00.954 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-16T10:10:00.954 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-16T10:10:00.954 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-16T10:10:00.954 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-16T10:10:00.954 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-16T10:10:00.954 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-16T10:10:00.955 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-16T10:10:00.955 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-16T10:10:00.955 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-16T10:10:00.955 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-16T10:10:00.955 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-16T10:10:00.955 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-16T10:10:00.955 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-16T10:10:00.955 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-16T10:10:00.955 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-16T10:10:00.955 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-16T10:10:00.955 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-16T10:10:00.955 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-16T10:10:00.955 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-16T10:10:00.955 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-16T10:10:00.956 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-16T10:10:00.956 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-16T10:10:00.956 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-16T10:10:00.956 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-16T10:10:00.956 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-16T10:10:00.956 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-16T10:10:00.956 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-16T10:10:00.956 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-16T10:10:00.956 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-16T10:10:00.956 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-16T10:10:00.956 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-16T10:10:00.956 INFO:tasks.ceph:Setting up mon.a... 2024-03-16T10:10:00.957 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-16T10:10:00.992 INFO:teuthology.orchestra.run.smithi017.stdout:creating /etc/ceph/ceph.keyring 2024-03-16T10:10:01.004 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-16T10:10:01.080 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-16T10:10:01.134 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.17'), ('mon.b', '172.21.15.174'), ('mon.c', '[v2:172.21.15.174:3301,v1:172.21.15.174:6790]')] 2024-03-16T10:10:01.135 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.17,172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.b': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-16T10:10:01.135 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:01.135 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-16T10:10:01.182 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.17 --add b 172.21.15.174 --addv c '[v2:172.21.15.174:3301,v1:172.21.15.174:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-16T10:10:01.270 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-16T10:10:01.270 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: generated fsid b21729d9-93de-4054-b789-790eefecfc1c 2024-03-16T10:10:01.270 INFO:teuthology.orchestra.run.smithi017.stdout:setting min_mon_release = quincy 2024-03-16T10:10:01.270 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 0 2024-03-16T10:10:01.270 INFO:teuthology.orchestra.run.smithi017.stdout:fsid b21729d9-93de-4054-b789-790eefecfc1c 2024-03-16T10:10:01.270 INFO:teuthology.orchestra.run.smithi017.stdout:last_changed 2024-03-16T10:10:01.272206+0000 2024-03-16T10:10:01.270 INFO:teuthology.orchestra.run.smithi017.stdout:created 2024-03-16T10:10:01.272206+0000 2024-03-16T10:10:01.270 INFO:teuthology.orchestra.run.smithi017.stdout:min_mon_release 17 (quincy) 2024-03-16T10:10:01.270 INFO:teuthology.orchestra.run.smithi017.stdout:election_strategy: 1 2024-03-16T10:10:01.271 INFO:teuthology.orchestra.run.smithi017.stdout:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-03-16T10:10:01.271 INFO:teuthology.orchestra.run.smithi017.stdout:1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.b 2024-03-16T10:10:01.271 INFO:teuthology.orchestra.run.smithi017.stdout:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2024-03-16T10:10:01.271 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-16T10:10:01.272 DEBUG:teuthology.orchestra.run.smithi017:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-16T10:10:01.276 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b21729d9-93de-4054-b789-790eefecfc1c... 2024-03-16T10:10:01.277 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-16T10:10:01.319 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-16T10:10:01.345 INFO:teuthology.orchestra.run.smithi017.stdout:[global] 2024-03-16T10:10:01.345 INFO:teuthology.orchestra.run.smithi017.stdout: chdir = "" 2024-03-16T10:10:01.345 INFO:teuthology.orchestra.run.smithi017.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: auth supported = cephx 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: filestore xattr use omap = true 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: mon clock drift allowed = 1.000 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: osd crush chooseleaf type = 0 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: auth debug = true 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on old message = true 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on bug = true 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: mon pg warn max object skew = 0 2024-03-16T10:10:01.346 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: # disable pg_autoscaler by default for new pools 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default size = 2 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow primary affinity = true 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow pg remap = true 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on legacy crush tunables = false 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on crush straw calc version zero = false 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on no sortbitwise = false 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on osd down out interval zero = false 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on too few osds = false 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-16T10:10:01.347 INFO:teuthology.orchestra.run.smithi017.stdout: mon_allow_pool_size_one = true 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: osd default data pool replay window = 5 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: mon allow pool delete = true 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: mon cluster log file level = debug 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: debug asserts on shutdown = true 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: mon health detail to clog = false 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: mon host = "172.21.15.17,172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790]" 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: fsid = b21729d9-93de-4054-b789-790eefecfc1c 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout:[osd] 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: osd journal size = 100 2024-03-16T10:10:01.348 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub load threshold = 5.0 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub max interval = 600 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: osd mclock profile = high_recovery_ops 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: osd recover clone overlap = true 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: osd recovery max chunk = 1048576 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug shutdown = true 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug op order = true 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug verify stray on activate = true 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug trim objects = true 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: osd open classes on start = true 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug pg log writeout = true 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.349 INFO:teuthology.orchestra.run.smithi017.stdout: osd deep scrub update digest min age = 30 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: osd map max advance = 10 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: journal zero on create = true 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: filestore ondisk finisher threads = 3 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: filestore apply finisher threads = 3 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: bdev debug aio = true 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug misdirected ops = true 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: bdev async discard = True 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: bdev enable discard = True 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore allocator = bitmap 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore block size = 96636764160 2024-03-16T10:10:01.350 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore fsck on mount = True 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluefs = 1/20 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluestore = 1/20 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: debug osd = 20 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: debug rocksdb = 4/10 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd backfillfull_ratio = 0.85 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd full ratio = 0.9 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd nearfull ratio = 0.8 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: osd failsafe full ratio = 0.95 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: osd objectstore = bluestore 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: osd op complaint time = 180 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr] 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: debug mgr = 20 2024-03-16T10:10:01.351 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: mgr/telemetry/nag = false 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout:[mon] 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: debug paxos = 20 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: mon data avail warn = 5 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: mon mgr mkfs grace = 240 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd reporter subtree level = osd 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd prime pg temp = true 2024-03-16T10:10:01.352 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2024-03-16T10:10:01.353 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.353 INFO:teuthology.orchestra.run.smithi017.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-16T10:10:01.353 INFO:teuthology.orchestra.run.smithi017.stdout: auth mon ticket ttl = 660 # 11m 2024-03-16T10:10:01.353 INFO:teuthology.orchestra.run.smithi017.stdout: auth service ticket ttl = 240 # 4m 2024-03-16T10:10:01.353 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.353 INFO:teuthology.orchestra.run.smithi017.stdout: # don't complain about insecure global_id in the test suite 2024-03-16T10:10:01.353 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-16T10:10:01.353 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-16T10:10:01.353 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.353 INFO:teuthology.orchestra.run.smithi017.stdout: # 1m isn't quite enough 2024-03-16T10:10:01.353 INFO:teuthology.orchestra.run.smithi017.stdout: mon_down_mkfs_grace = 2m 2024-03-16T10:10:01.353 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.353 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_filestore_osds = false 2024-03-16T10:10:01.353 INFO:teuthology.orchestra.run.smithi017.stdout: mon op complaint time = 120 2024-03-16T10:10:01.354 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:01.354 INFO:teuthology.orchestra.run.smithi017.stdout:[client] 2024-03-16T10:10:01.354 INFO:teuthology.orchestra.run.smithi017.stdout: rgw cache enabled = true 2024-03-16T10:10:01.354 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable ops log = true 2024-03-16T10:10:01.354 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable usage log = true 2024-03-16T10:10:01.354 INFO:teuthology.orchestra.run.smithi017.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-16T10:10:01.354 INFO:teuthology.orchestra.run.smithi017.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-16T10:10:01.354 INFO:teuthology.orchestra.run.smithi017.stdout: client mount timeout = 600 2024-03-16T10:10:01.354 INFO:teuthology.orchestra.run.smithi017.stdout: debug client = 20 2024-03-16T10:10:01.354 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-03-16T10:10:01.354 INFO:teuthology.orchestra.run.smithi017.stdout: pid file = "" 2024-03-16T10:10:01.354 INFO:teuthology.orchestra.run.smithi017.stdout: rados mon op timeout = 900 2024-03-16T10:10:01.357 INFO:teuthology.orchestra.run.smithi017.stdout: rados osd op timeout = 900 2024-03-16T10:10:01.357 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.a] 2024-03-16T10:10:01.357 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.b] 2024-03-16T10:10:01.357 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.c] 2024-03-16T10:10:01.357 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.a] 2024-03-16T10:10:01.357 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.b] 2024-03-16T10:10:01.357 INFO:teuthology.orchestra.run.smithi017.stdout:[mds] 2024-03-16T10:10:01.357 INFO:teuthology.orchestra.run.smithi017.stdout: debug mds = 20 2024-03-16T10:10:01.357 INFO:teuthology.orchestra.run.smithi017.stdout: debug mds balancer = 20 2024-03-16T10:10:01.357 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-03-16T10:10:01.357 INFO:teuthology.orchestra.run.smithi017.stdout: mds debug frag = True 2024-03-16T10:10:01.358 INFO:teuthology.orchestra.run.smithi017.stdout: mds debug scatterstat = True 2024-03-16T10:10:01.358 INFO:teuthology.orchestra.run.smithi017.stdout: mds op complaint time = 180 2024-03-16T10:10:01.358 INFO:teuthology.orchestra.run.smithi017.stdout: mds verify scatter = True 2024-03-16T10:10:01.358 INFO:teuthology.orchestra.run.smithi017.stdout: osd op complaint time = 180 2024-03-16T10:10:01.358 INFO:teuthology.orchestra.run.smithi017.stdout: rados mon op timeout = 900 2024-03-16T10:10:01.358 INFO:teuthology.orchestra.run.smithi017.stdout: rados osd op timeout = 900 2024-03-16T10:10:01.358 INFO:teuthology.orchestra.run.smithi174.stdout:[global] 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: chdir = "" 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: auth supported = cephx 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: filestore xattr use omap = true 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: mon clock drift allowed = 1.000 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: osd crush chooseleaf type = 0 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: auth debug = true 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on old message = true 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: ms die on bug = true 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.359 INFO:teuthology.orchestra.run.smithi174.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-16T10:10:01.360 INFO:teuthology.orchestra.run.smithi174.stdout: mon pg warn max object skew = 0 2024-03-16T10:10:01.360 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.360 INFO:teuthology.orchestra.run.smithi174.stdout: # disable pg_autoscaler by default for new pools 2024-03-16T10:10:01.360 INFO:teuthology.orchestra.run.smithi174.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-16T10:10:01.360 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.360 INFO:teuthology.orchestra.run.smithi174.stdout: osd pool default size = 2 2024-03-16T10:10:01.360 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.360 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd allow primary affinity = true 2024-03-16T10:10:01.360 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd allow pg remap = true 2024-03-16T10:10:01.360 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on legacy crush tunables = false 2024-03-16T10:10:01.360 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on crush straw calc version zero = false 2024-03-16T10:10:01.360 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on no sortbitwise = false 2024-03-16T10:10:01.360 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on osd down out interval zero = false 2024-03-16T10:10:01.360 INFO:teuthology.orchestra.run.smithi174.stdout: mon warn on too few osds = false 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: mon_allow_pool_size_one = true 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: osd default data pool replay window = 5 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: mon allow pool delete = true 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: mon cluster log file level = debug 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: debug asserts on shutdown = true 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: mon health detail to clog = false 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: mon host = "172.21.15.17,172.21.15.174,[v2:172.21.15.174:3301,v1:172.21.15.174:6790]" 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: fsid = b21729d9-93de-4054-b789-790eefecfc1c 2024-03-16T10:10:01.361 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.362 INFO:teuthology.orchestra.run.smithi174.stdout:[osd] 2024-03-16T10:10:01.362 INFO:teuthology.orchestra.run.smithi174.stdout: osd journal size = 100 2024-03-16T10:10:01.362 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.362 INFO:teuthology.orchestra.run.smithi174.stdout: osd scrub load threshold = 5.0 2024-03-16T10:10:01.362 INFO:teuthology.orchestra.run.smithi174.stdout: osd scrub max interval = 600 2024-03-16T10:10:01.362 INFO:teuthology.orchestra.run.smithi174.stdout: osd mclock profile = high_recovery_ops 2024-03-16T10:10:01.362 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.362 INFO:teuthology.orchestra.run.smithi174.stdout: osd recover clone overlap = true 2024-03-16T10:10:01.362 INFO:teuthology.orchestra.run.smithi174.stdout: osd recovery max chunk = 1048576 2024-03-16T10:10:01.362 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.362 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug shutdown = true 2024-03-16T10:10:01.362 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug op order = true 2024-03-16T10:10:01.362 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug verify stray on activate = true 2024-03-16T10:10:01.362 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug trim objects = true 2024-03-16T10:10:01.363 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.363 INFO:teuthology.orchestra.run.smithi174.stdout: osd open classes on start = true 2024-03-16T10:10:01.363 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug pg log writeout = true 2024-03-16T10:10:01.363 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.363 INFO:teuthology.orchestra.run.smithi174.stdout: osd deep scrub update digest min age = 30 2024-03-16T10:10:01.363 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.363 INFO:teuthology.orchestra.run.smithi174.stdout: osd map max advance = 10 2024-03-16T10:10:01.363 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.363 INFO:teuthology.orchestra.run.smithi174.stdout: journal zero on create = true 2024-03-16T10:10:01.363 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.363 INFO:teuthology.orchestra.run.smithi174.stdout: filestore ondisk finisher threads = 3 2024-03-16T10:10:01.363 INFO:teuthology.orchestra.run.smithi174.stdout: filestore apply finisher threads = 3 2024-03-16T10:10:01.364 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.364 INFO:teuthology.orchestra.run.smithi174.stdout: bdev debug aio = true 2024-03-16T10:10:01.364 INFO:teuthology.orchestra.run.smithi174.stdout: osd debug misdirected ops = true 2024-03-16T10:10:01.364 INFO:teuthology.orchestra.run.smithi174.stdout: bdev async discard = True 2024-03-16T10:10:01.364 INFO:teuthology.orchestra.run.smithi174.stdout: bdev enable discard = True 2024-03-16T10:10:01.364 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore allocator = bitmap 2024-03-16T10:10:01.365 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore block size = 96636764160 2024-03-16T10:10:01.365 INFO:teuthology.orchestra.run.smithi174.stdout: bluestore fsck on mount = True 2024-03-16T10:10:01.365 INFO:teuthology.orchestra.run.smithi174.stdout: debug bluefs = 1/20 2024-03-16T10:10:01.365 INFO:teuthology.orchestra.run.smithi174.stdout: debug bluestore = 1/20 2024-03-16T10:10:01.365 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2024-03-16T10:10:01.365 INFO:teuthology.orchestra.run.smithi174.stdout: debug osd = 20 2024-03-16T10:10:01.365 INFO:teuthology.orchestra.run.smithi174.stdout: debug rocksdb = 4/10 2024-03-16T10:10:01.365 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd backfillfull_ratio = 0.85 2024-03-16T10:10:01.366 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd full ratio = 0.9 2024-03-16T10:10:01.366 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd nearfull ratio = 0.8 2024-03-16T10:10:01.366 INFO:teuthology.orchestra.run.smithi174.stdout: osd failsafe full ratio = 0.95 2024-03-16T10:10:01.366 INFO:teuthology.orchestra.run.smithi174.stdout: osd objectstore = bluestore 2024-03-16T10:10:01.366 INFO:teuthology.orchestra.run.smithi174.stdout: osd op complaint time = 180 2024-03-16T10:10:01.366 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.366 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr] 2024-03-16T10:10:01.367 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2024-03-16T10:10:01.367 INFO:teuthology.orchestra.run.smithi174.stdout: debug mgr = 20 2024-03-16T10:10:01.367 INFO:teuthology.orchestra.run.smithi174.stdout: debug mon = 20 2024-03-16T10:10:01.367 INFO:teuthology.orchestra.run.smithi174.stdout: debug auth = 20 2024-03-16T10:10:01.367 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min pgs per osd = 4 2024-03-16T10:10:01.367 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min bytes per osd = 10 2024-03-16T10:10:01.368 INFO:teuthology.orchestra.run.smithi174.stdout: mgr/telemetry/nag = false 2024-03-16T10:10:01.368 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.368 INFO:teuthology.orchestra.run.smithi174.stdout:[mon] 2024-03-16T10:10:01.368 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2024-03-16T10:10:01.368 INFO:teuthology.orchestra.run.smithi174.stdout: debug mon = 20 2024-03-16T10:10:01.369 INFO:teuthology.orchestra.run.smithi174.stdout: debug paxos = 20 2024-03-16T10:10:01.369 INFO:teuthology.orchestra.run.smithi174.stdout: debug auth = 20 2024-03-16T10:10:01.369 INFO:teuthology.orchestra.run.smithi174.stdout: mon data avail warn = 5 2024-03-16T10:10:01.369 INFO:teuthology.orchestra.run.smithi174.stdout: mon mgr mkfs grace = 240 2024-03-16T10:10:01.369 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min pgs per osd = 4 2024-03-16T10:10:01.369 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd reporter subtree level = osd 2024-03-16T10:10:01.370 INFO:teuthology.orchestra.run.smithi174.stdout: mon osd prime pg temp = true 2024-03-16T10:10:01.370 INFO:teuthology.orchestra.run.smithi174.stdout: mon reweight min bytes per osd = 10 2024-03-16T10:10:01.370 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.370 INFO:teuthology.orchestra.run.smithi174.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-16T10:10:01.370 INFO:teuthology.orchestra.run.smithi174.stdout: auth mon ticket ttl = 660 # 11m 2024-03-16T10:10:01.370 INFO:teuthology.orchestra.run.smithi174.stdout: auth service ticket ttl = 240 # 4m 2024-03-16T10:10:01.371 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.371 INFO:teuthology.orchestra.run.smithi174.stdout: # don't complain about insecure global_id in the test suite 2024-03-16T10:10:01.371 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-16T10:10:01.371 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-16T10:10:01.371 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.371 INFO:teuthology.orchestra.run.smithi174.stdout: # 1m isn't quite enough 2024-03-16T10:10:01.372 INFO:teuthology.orchestra.run.smithi174.stdout: mon_down_mkfs_grace = 2m 2024-03-16T10:10:01.372 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.372 INFO:teuthology.orchestra.run.smithi174.stdout: mon_warn_on_filestore_osds = false 2024-03-16T10:10:01.372 INFO:teuthology.orchestra.run.smithi174.stdout: mon op complaint time = 120 2024-03-16T10:10:01.372 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-03-16T10:10:01.372 INFO:teuthology.orchestra.run.smithi174.stdout:[client] 2024-03-16T10:10:01.373 INFO:teuthology.orchestra.run.smithi174.stdout: rgw cache enabled = true 2024-03-16T10:10:01.373 INFO:teuthology.orchestra.run.smithi174.stdout: rgw enable ops log = true 2024-03-16T10:10:01.373 INFO:teuthology.orchestra.run.smithi174.stdout: rgw enable usage log = true 2024-03-16T10:10:01.373 INFO:teuthology.orchestra.run.smithi174.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-16T10:10:01.373 INFO:teuthology.orchestra.run.smithi174.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-16T10:10:01.373 INFO:teuthology.orchestra.run.smithi174.stdout: client mount timeout = 600 2024-03-16T10:10:01.374 INFO:teuthology.orchestra.run.smithi174.stdout: debug client = 20 2024-03-16T10:10:01.374 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2024-03-16T10:10:01.374 INFO:teuthology.orchestra.run.smithi174.stdout: pid file = "" 2024-03-16T10:10:01.374 INFO:teuthology.orchestra.run.smithi174.stdout: rados mon op timeout = 900 2024-03-16T10:10:01.374 INFO:teuthology.orchestra.run.smithi174.stdout: rados osd op timeout = 900 2024-03-16T10:10:01.374 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.a] 2024-03-16T10:10:01.375 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.b] 2024-03-16T10:10:01.375 INFO:teuthology.orchestra.run.smithi174.stdout:[mon.c] 2024-03-16T10:10:01.375 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.a] 2024-03-16T10:10:01.375 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.b] 2024-03-16T10:10:01.375 INFO:teuthology.orchestra.run.smithi174.stdout:[mds] 2024-03-16T10:10:01.375 INFO:teuthology.orchestra.run.smithi174.stdout: debug mds = 20 2024-03-16T10:10:01.376 INFO:teuthology.orchestra.run.smithi174.stdout: debug mds balancer = 20 2024-03-16T10:10:01.376 INFO:teuthology.orchestra.run.smithi174.stdout: debug ms = 1 2024-03-16T10:10:01.376 INFO:teuthology.orchestra.run.smithi174.stdout: mds debug frag = True 2024-03-16T10:10:01.376 INFO:teuthology.orchestra.run.smithi174.stdout: mds debug scatterstat = True 2024-03-16T10:10:01.376 INFO:teuthology.orchestra.run.smithi174.stdout: mds op complaint time = 180 2024-03-16T10:10:01.376 INFO:teuthology.orchestra.run.smithi174.stdout: mds verify scatter = True 2024-03-16T10:10:01.377 INFO:teuthology.orchestra.run.smithi174.stdout: osd op complaint time = 180 2024-03-16T10:10:01.377 INFO:teuthology.orchestra.run.smithi174.stdout: rados mon op timeout = 900 2024-03-16T10:10:01.377 INFO:teuthology.orchestra.run.smithi174.stdout: rados osd op timeout = 900 2024-03-16T10:10:01.379 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-16T10:10:01.380 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-16T10:10:01.475 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-16T10:10:01.475 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:01.475 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-16T10:10:01.480 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:01.480 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-16T10:10:01.525 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi017.front.sepia.ceph.com 2024-03-16T10:10:01.525 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:01.525 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-16T10:10:01.526 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-16T10:10:01.590 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:01.590 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-16T10:10:01.642 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi174.front.sepia.ceph.com 2024-03-16T10:10:01.642 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:10:01.642 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-16T10:10:01.642 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-16T10:10:01.666 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:10:01.666 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-16T10:10:01.715 INFO:tasks.ceph:Setting up mon nodes... 2024-03-16T10:10:01.715 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-16T10:10:01.715 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-16T10:10:01.759 INFO:teuthology.orchestra.run.smithi017.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-16T10:10:01.762 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-16T10:10:01.806 INFO:teuthology.orchestra.run.smithi174.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-16T10:10:01.822 INFO:tasks.ceph:Setting up mds nodes... 2024-03-16T10:10:01.822 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-16T10:10:01.867 INFO:teuthology.orchestra.run.smithi017.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-16T10:10:01.870 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-16T10:10:01.882 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-16T10:10:01.923 INFO:teuthology.orchestra.run.smithi174.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-16T10:10:01.926 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-16T10:10:01.981 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-16T10:10:01.981 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-16T10:10:02.018 INFO:teuthology.orchestra.run.smithi017.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-16T10:10:02.030 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-16T10:10:02.030 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi017.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi174.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-16T10:10:02.030 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-16T10:10:02.091 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T10:10:02.091 INFO:tasks.ceph:role: osd.0 2024-03-16T10:10:02.091 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-16T10:10:02.091 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-16T10:10:02.146 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T10:10:02.147 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T10:10:02.147 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T10:10:02.147 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T10:10:02.147 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T10:10:02.147 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-03-16T10:10:02.147 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T10:10:02.147 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T10:10:02.147 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T10:10:02.147 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T10:10:02.170 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-03-16T10:10:02.172 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-03-16T10:10:02.172 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-16T10:10:02.230 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-16T10:10:02.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:10:02.283 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T10:10:02.283 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-16T10:10:02.335 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T10:10:02.335 INFO:tasks.ceph:role: osd.1 2024-03-16T10:10:02.335 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-16T10:10:02.335 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-16T10:10:02.387 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T10:10:02.388 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T10:10:02.388 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T10:10:02.388 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T10:10:02.388 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T10:10:02.388 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-03-16T10:10:02.388 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T10:10:02.388 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T10:10:02.388 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T10:10:02.388 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T10:10:02.412 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-03-16T10:10:02.413 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-03-16T10:10:02.413 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-16T10:10:02.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-16T10:10:02.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:10:02.533 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T10:10:02.533 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-16T10:10:02.590 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T10:10:02.590 INFO:tasks.ceph:role: osd.2 2024-03-16T10:10:02.590 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-16T10:10:02.590 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-16T10:10:02.646 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T10:10:02.646 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T10:10:02.647 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T10:10:02.647 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T10:10:02.647 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T10:10:02.647 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-03-16T10:10:02.647 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T10:10:02.647 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T10:10:02.647 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T10:10:02.647 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T10:10:02.672 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-03-16T10:10:02.674 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-03-16T10:10:02.674 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-16T10:10:02.738 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-16T10:10:02.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:10:02.792 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T10:10:02.792 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-16T10:10:02.845 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T10:10:02.845 INFO:tasks.ceph:role: osd.3 2024-03-16T10:10:02.845 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-16T10:10:02.845 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-16T10:10:02.899 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T10:10:02.899 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T10:10:02.899 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T10:10:02.899 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T10:10:02.900 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T10:10:02.900 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-03-16T10:10:02.900 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T10:10:02.900 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T10:10:02.900 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T10:10:02.900 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T10:10:02.924 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-03-16T10:10:02.925 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-03-16T10:10:02.925 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-16T10:10:02.986 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-16T10:10:03.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:10:03.039 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T10:10:03.039 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T10:10:03.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:03.143+0000 7f87885077c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-16T10:10:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:03.143+0000 7f87885077c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-16T10:10:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:03.143+0000 7f87885077c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-16T10:10:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:03.143+0000 7f87885077c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-16T10:10:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:03.143+0000 7f87885077c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-16T10:10:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:03.143+0000 7f87885077c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-16T10:10:06.117 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-16T10:10:06.131 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T10:10:06.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:06.215+0000 7f7c49c1e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-16T10:10:06.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:06.215+0000 7f7c49c1e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-16T10:10:06.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:06.215+0000 7f7c49c1e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-16T10:10:06.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:06.215+0000 7f7c49c1e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-16T10:10:06.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:06.215+0000 7f7c49c1e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-16T10:10:06.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:06.219+0000 7f7c49c1e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-16T10:10:09.197 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-16T10:10:09.251 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T10:10:09.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:09.335+0000 7f4e4a91d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-16T10:10:09.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:09.335+0000 7f4e4a91d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-16T10:10:09.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:09.335+0000 7f4e4a91d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-16T10:10:09.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:09.335+0000 7f4e4a91d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-16T10:10:09.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:09.335+0000 7f4e4a91d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-16T10:10:09.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:09.335+0000 7f4e4a91d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-16T10:10:12.304 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-16T10:10:12.359 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T10:10:12.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:12.443+0000 7fae0f4ff7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-16T10:10:12.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:12.443+0000 7fae0f4ff7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-16T10:10:12.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:12.443+0000 7fae0f4ff7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-16T10:10:12.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:12.443+0000 7fae0f4ff7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-16T10:10:12.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:12.443+0000 7fae0f4ff7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-16T10:10:12.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:12.443+0000 7fae0f4ff7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-16T10:10:15.421 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-16T10:10:15.474 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-16T10:10:15.489 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T10:10:15.489 INFO:tasks.ceph:role: osd.4 2024-03-16T10:10:15.490 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi174.front.sepia.ceph.com 2024-03-16T10:10:15.490 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-16T10:10:15.550 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T10:10:15.551 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T10:10:15.551 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T10:10:15.551 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T10:10:15.551 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T10:10:15.551 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2024-03-16T10:10:15.551 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T10:10:15.551 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T10:10:15.551 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T10:10:15.551 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T10:10:16.027 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2024-03-16T10:10:16.029 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2024-03-16T10:10:16.029 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-16T10:10:16.189 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-16T10:10:16.200 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T10:10:16.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:10:16.201 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-16T10:10:16.258 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T10:10:16.258 INFO:tasks.ceph:role: osd.5 2024-03-16T10:10:16.258 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi174.front.sepia.ceph.com 2024-03-16T10:10:16.258 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-16T10:10:16.326 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T10:10:16.326 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T10:10:16.326 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T10:10:16.326 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T10:10:16.326 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T10:10:16.326 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2024-03-16T10:10:16.326 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T10:10:16.326 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T10:10:16.326 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T10:10:16.326 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T10:10:16.771 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2024-03-16T10:10:16.773 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2024-03-16T10:10:16.773 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-16T10:10:16.857 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-16T10:10:16.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:10:16.907 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T10:10:16.907 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-16T10:10:16.960 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T10:10:16.960 INFO:tasks.ceph:role: osd.6 2024-03-16T10:10:16.961 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi174.front.sepia.ceph.com 2024-03-16T10:10:16.961 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-16T10:10:17.023 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T10:10:17.023 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T10:10:17.023 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T10:10:17.023 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T10:10:17.023 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T10:10:17.023 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2024-03-16T10:10:17.024 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T10:10:17.024 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T10:10:17.024 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T10:10:17.024 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T10:10:17.486 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2024-03-16T10:10:17.488 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2024-03-16T10:10:17.489 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-16T10:10:17.621 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-16T10:10:17.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:10:17.633 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T10:10:17.633 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-16T10:10:17.686 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T10:10:17.686 INFO:tasks.ceph:role: osd.7 2024-03-16T10:10:17.686 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi174.front.sepia.ceph.com 2024-03-16T10:10:17.686 DEBUG:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-16T10:10:17.753 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T10:10:17.753 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T10:10:17.753 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T10:10:17.753 INFO:teuthology.orchestra.run.smithi174.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T10:10:17.753 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T10:10:17.753 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2024-03-16T10:10:17.753 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T10:10:17.753 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T10:10:17.753 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T10:10:17.753 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T10:10:18.257 INFO:teuthology.orchestra.run.smithi174.stdout:Discarding blocks...Done. 2024-03-16T10:10:18.260 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2024-03-16T10:10:18.260 DEBUG:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-16T10:10:18.395 DEBUG:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-16T10:10:18.406 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T10:10:18.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:10:18.408 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T10:10:18.523 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:18.522+0000 7f42030bd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-16T10:10:18.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:18.526+0000 7f42030bd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-16T10:10:18.526 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:18.526+0000 7f42030bd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-16T10:10:18.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:18.526+0000 7f42030bd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-16T10:10:18.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:18.526+0000 7f42030bd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-16T10:10:18.527 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:18.526+0000 7f42030bd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-16T10:10:21.496 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-16T10:10:21.511 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T10:10:21.592 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:21.590+0000 7f36704867c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-16T10:10:21.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:21.590+0000 7f36704867c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-16T10:10:21.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:21.590+0000 7f36704867c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-16T10:10:21.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:21.590+0000 7f36704867c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-16T10:10:21.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:21.590+0000 7f36704867c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-16T10:10:21.593 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:21.590+0000 7f36704867c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-16T10:10:24.561 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-16T10:10:24.615 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T10:10:24.697 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:24.694+0000 7f783a1437c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-16T10:10:24.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:24.694+0000 7f783a1437c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-16T10:10:24.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:24.694+0000 7f783a1437c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-16T10:10:24.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:24.694+0000 7f783a1437c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-16T10:10:24.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:24.694+0000 7f783a1437c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-16T10:10:24.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:24.698+0000 7f783a1437c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-16T10:10:27.670 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-16T10:10:27.727 DEBUG:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T10:10:27.813 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:27.810+0000 7fe476beb7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-16T10:10:27.814 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:27.810+0000 7fe476beb7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-16T10:10:27.814 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:27.810+0000 7fe476beb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-16T10:10:27.814 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:27.810+0000 7fe476beb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-16T10:10:27.814 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:27.810+0000 7fe476beb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-16T10:10:27.815 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:27.814+0000 7fe476beb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-16T10:10:29.619 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T10:10:29.622 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T10:10:30.781 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-16T10:10:30.795 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-16T10:10:30.795 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:30.795 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-16T10:10:30.809 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:30.809 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-16T10:10:30.866 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:30.866 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-16T10:10:30.921 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:30.922 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-16T10:10:30.978 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:30.978 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-16T10:10:31.034 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:31.034 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-16T10:10:31.090 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:10:31.090 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-16T10:10:31.104 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:10:31.104 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-16T10:10:31.159 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:10:31.159 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-16T10:10:31.213 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:10:31.213 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-16T10:10:31.269 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:10:31.270 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-16T10:10:31.321 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:10:31.321 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-16T10:10:31.373 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:31.373 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-16T10:10:31.379 INFO:tasks.ceph:Adding keys to all mons... 2024-03-16T10:10:31.379 DEBUG:teuthology.orchestra.run.smithi017:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-16T10:10:31.423 DEBUG:teuthology.orchestra.run.smithi174:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-16T10:10:31.433 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr.y] 2024-03-16T10:10:31.433 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD5b/Vlv4ZiLRAAsPpAwuAQsda6/9P0ltUIPQ== 2024-03-16T10:10:31.433 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.a] 2024-03-16T10:10:31.433 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD5b/VlioLRMxAA8TL1+mZbA/7VcCJxYTFtww== 2024-03-16T10:10:31.434 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.0] 2024-03-16T10:10:31.434 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD7b/VlNBu6CBAAzHRY4hnKP3VMVq6nCYXDKw== 2024-03-16T10:10:31.434 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.1] 2024-03-16T10:10:31.434 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD+b/Vl1WUSDRAAUC0iQzl7b7N/4tbm6Vm3+w== 2024-03-16T10:10:31.434 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.2] 2024-03-16T10:10:31.434 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQABcPVlpZwMFBAA7/Ksf/Of6feuG6+1tI4cag== 2024-03-16T10:10:31.434 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.3] 2024-03-16T10:10:31.434 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQAEcPVl70SBGhAAymTiXwnBJPeopd5TS5vsIg== 2024-03-16T10:10:31.434 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr.x] 2024-03-16T10:10:31.434 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD5b/VlC7UbMBAAmnD6q7ZwPqoXjZfA+iVK9A== 2024-03-16T10:10:31.434 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.b] 2024-03-16T10:10:31.434 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD5b/VlOUAVNxAAGphuN1rjaCdQcWq+gH79Gg== 2024-03-16T10:10:31.434 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.4] 2024-03-16T10:10:31.435 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQAKcPVlTm5CHxAA32mdxIATfNlJUbppaEYvNg== 2024-03-16T10:10:31.435 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.5] 2024-03-16T10:10:31.435 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQANcPVl4phgIxAAjlG3iIismsV4ivhIFSjM1g== 2024-03-16T10:10:31.435 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.6] 2024-03-16T10:10:31.435 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQAQcPVlV0ykKRAAFTAAIf4cgeITuayHHi+B8w== 2024-03-16T10:10:31.435 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.7] 2024-03-16T10:10:31.435 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQATcPVlkOSNMBAA4zy6wXASGhCKcXPrjEcpZA== 2024-03-16T10:10:31.435 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-16T10:10:31.435 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD6b/Vl9GM/ARAAE2ObwC6ro6fuTOmqZ2GNzg== 2024-03-16T10:10:31.436 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr.y] 2024-03-16T10:10:31.436 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQD5b/Vlv4ZiLRAAsPpAwuAQsda6/9P0ltUIPQ== 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.a] 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQD5b/VlioLRMxAA8TL1+mZbA/7VcCJxYTFtww== 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.0] 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQD7b/VlNBu6CBAAzHRY4hnKP3VMVq6nCYXDKw== 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.1] 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQD+b/Vl1WUSDRAAUC0iQzl7b7N/4tbm6Vm3+w== 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.2] 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQABcPVlpZwMFBAA7/Ksf/Of6feuG6+1tI4cag== 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.3] 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAEcPVl70SBGhAAymTiXwnBJPeopd5TS5vsIg== 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr.x] 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQD5b/VlC7UbMBAAmnD6q7ZwPqoXjZfA+iVK9A== 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout:[mds.b] 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQD5b/VlOUAVNxAAGphuN1rjaCdQcWq+gH79Gg== 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.4] 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAKcPVlTm5CHxAA32mdxIATfNlJUbppaEYvNg== 2024-03-16T10:10:31.437 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.5] 2024-03-16T10:10:31.438 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQANcPVl4phgIxAAjlG3iIismsV4ivhIFSjM1g== 2024-03-16T10:10:31.438 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.6] 2024-03-16T10:10:31.438 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAQcPVlV0ykKRAAFTAAIf4cgeITuayHHi+B8w== 2024-03-16T10:10:31.438 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.7] 2024-03-16T10:10:31.438 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQATcPVlkOSNMBAA4zy6wXASGhCKcXPrjEcpZA== 2024-03-16T10:10:31.438 INFO:teuthology.orchestra.run.smithi174.stdout:[client.0] 2024-03-16T10:10:31.438 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQD6b/Vl9GM/ARAAE2ObwC6ro6fuTOmqZ2GNzg== 2024-03-16T10:10:31.438 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-16T10:10:31.479 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-16T10:10:31.524 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T10:10:31.526 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T10:10:31.614 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:31.617 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:31.656 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:31.699 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:31.738 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:31.783 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:31.824 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:31.867 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:31.907 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-16T10:10:31.951 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-16T10:10:31.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T10:10:32.035 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T10:10:32.078 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:32.080 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:32.119 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:32.163 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:32.207 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:32.209 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:32.284 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:32.291 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T10:10:32.371 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-16T10:10:32.375 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-16T10:10:32.423 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-16T10:10:32.423 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-16T10:10:32.435 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-16T10:10:32.523 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-16T10:10:32.575 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-16T10:10:32.589 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-16T10:10:32.681 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-16T10:10:32.691 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-16T10:10:32.743 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-16T10:10:32.832 DEBUG:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-16T10:10:32.842 DEBUG:teuthology.orchestra.run.smithi017:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-16T10:10:32.844 DEBUG:teuthology.orchestra.run.smithi174:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-16T10:10:32.889 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-16T10:10:32.890 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-16T10:10:32.890 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-16T10:10:32.892 INFO:tasks.ceph.mon.a:Started 2024-03-16T10:10:32.892 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-16T10:10:32.892 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-16T10:10:32.931 INFO:tasks.ceph.mon.b:Started 2024-03-16T10:10:32.931 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-16T10:10:32.931 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-16T10:10:32.933 INFO:tasks.ceph.mon.c:Started 2024-03-16T10:10:32.933 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-16T10:10:32.933 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-16T10:10:32.934 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-16T10:10:32.936 INFO:tasks.ceph.mgr.y:Started 2024-03-16T10:10:32.936 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-16T10:10:32.936 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-16T10:10:32.938 INFO:tasks.ceph.mgr.x:Started 2024-03-16T10:10:32.938 INFO:tasks.ceph:Setting crush tunables to default 2024-03-16T10:10:32.938 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph osd crush tunables default 2024-03-16T10:10:33.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bfa4c640 1 Processor -- start 2024-03-16T10:10:33.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bfa4c640 1 -- start start 2024-03-16T10:10:33.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bfa4c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 0x7fe4b81410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:33.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bfa4c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe4b80b7b70 0x7fe4b8141630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:33.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bfa4c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b806ce50 con 0x7fe4b80b63a0 2024-03-16T10:10:33.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bfa4c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b80b6780 con 0x7fe4b80b7b70 2024-03-16T10:10:33.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bfa4c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b80b6cc0 con 0x7fe4b8073070 2024-03-16T10:10:33.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bd7e8640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe4b80b63a0 0x7fe4b806d270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52812/0 (socket says 172.21.15.17:52812) 2024-03-16T10:10:33.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bd7e8640 1 -- 172.21.15.17:0/2759077671 learned_addr learned my addr 172.21.15.17:0/2759077671 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:10:33.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bdfe9640 1 --2- 172.21.15.17:0/2759077671 >> v2:172.21.15.17:3300/0 conn(0x7fe4b80b7b70 0x7fe4b8141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bdfe9640 1 -- 172.21.15.17:0/2759077671 >> v2:172.21.15.17:3300/0 conn(0x7fe4b80b7b70 msgr2=0x7fe4b8141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-16T10:10:33.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bdfe9640 1 -- 172.21.15.17:0/2759077671 >> v2:172.21.15.17:3300/0 conn(0x7fe4b80b7b70 msgr2=0x7fe4b8141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-16T10:10:33.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bdfe9640 1 --2- 172.21.15.17:0/2759077671 >> v2:172.21.15.17:3300/0 conn(0x7fe4b80b7b70 0x7fe4b8141630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T10:10:33.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bcfe7640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 0x7fe4b81410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bdfe9640 1 --2- 172.21.15.17:0/2759077671 >> v2:172.21.15.17:3300/0 conn(0x7fe4b80b7b70 0x7fe4b8141630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:10:33.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bcfe7640 1 -- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 msgr2=0x7fe4b81410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-16T10:10:33.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bcfe7640 1 -- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 msgr2=0x7fe4b81410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-16T10:10:33.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bcfe7640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 0x7fe4b81410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T10:10:33.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.058+0000 7fe4bcfe7640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 0x7fe4b81410f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T10:10:33.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.262+0000 7fe4bdfe9640 1 --2- 172.21.15.17:0/2759077671 >> v2:172.21.15.17:3300/0 conn(0x7fe4b80b7b70 0x7fe4b8141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.262+0000 7fe4bdfe9640 1 -- 172.21.15.17:0/2759077671 >> v2:172.21.15.17:3300/0 conn(0x7fe4b80b7b70 msgr2=0x7fe4b8141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-16T10:10:33.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.262+0000 7fe4bdfe9640 1 -- 172.21.15.17:0/2759077671 >> v2:172.21.15.17:3300/0 conn(0x7fe4b80b7b70 msgr2=0x7fe4b8141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-16T10:10:33.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.262+0000 7fe4bdfe9640 1 --2- 172.21.15.17:0/2759077671 >> v2:172.21.15.17:3300/0 conn(0x7fe4b80b7b70 0x7fe4b8141630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T10:10:33.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.262+0000 7fe4bcfe7640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 0x7fe4b81410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.262+0000 7fe4bdfe9640 1 --2- 172.21.15.17:0/2759077671 >> v2:172.21.15.17:3300/0 conn(0x7fe4b80b7b70 0x7fe4b8141630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T10:10:33.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.262+0000 7fe4bcfe7640 1 -- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 msgr2=0x7fe4b81410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-16T10:10:33.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.262+0000 7fe4bcfe7640 1 -- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 msgr2=0x7fe4b81410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-16T10:10:33.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.262+0000 7fe4bcfe7640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 0x7fe4b81410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T10:10:33.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.262+0000 7fe4bcfe7640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 0x7fe4b81410f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T10:10:33.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 -- 172.21.15.17:0/2759077671 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2067075694 0 0) 0x7fe4b806ce50 con 0x7fe4b80b63a0 2024-03-16T10:10:33.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 -- 172.21.15.17:0/2759077671 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4a4003530 con 0x7fe4b80b63a0 2024-03-16T10:10:33.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 -- 172.21.15.17:0/2759077671 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 155698923 0 0) 0x7fe4a4003530 con 0x7fe4b80b63a0 2024-03-16T10:10:33.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 -- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 msgr2=0x7fe4b81410f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-16T10:10:33.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 0x7fe4b81410f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 -- 172.21.15.17:0/2759077671 >> v2:172.21.15.17:3300/0 conn(0x7fe4b80b7b70 msgr2=0x7fe4b8141630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-16T10:10:33.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 --2- 172.21.15.17:0/2759077671 >> v2:172.21.15.17:3300/0 conn(0x7fe4b80b7b70 0x7fe4b8141630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 -- 172.21.15.17:0/2759077671 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b80b6fa0 con 0x7fe4b80b63a0 2024-03-16T10:10:33.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 -- 172.21.15.17:0/2759077671 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fe4b0001a00 con 0x7fe4b80b63a0 2024-03-16T10:10:33.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 -- 172.21.15.17:0/2759077671 >> v1:172.21.15.17:6789/0 conn(0x7fe4b80b63a0 legacy=0x7fe4b806d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:33.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 0x7fe4a40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:33.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4a4003ff0 0x7fe4a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:33.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4a4004d50 0x7fe4a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:33.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 -- 172.21.15.17:0/2759077671 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b80b6780 con 0x7fe4a4003ff0 2024-03-16T10:10:33.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 -- 172.21.15.17:0/2759077671 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b80b6cc0 con 0x7fe4a4004d50 2024-03-16T10:10:33.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4affff640 1 -- 172.21.15.17:0/2759077671 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a400bb90 con 0x7fe4b8073070 2024-03-16T10:10:33.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4bcfe7640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4a4003ff0 0x7fe4a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4bd7e8640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 0x7fe4a40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4bdfe9640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4a4004d50 0x7fe4a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4bdfe9640 1 -- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 msgr2=0x7fe4a40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:33.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4bdfe9640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4b8073070 0x7fe4a40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4bdfe9640 1 -- 172.21.15.17:0/2759077671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4a4003ff0 msgr2=0x7fe4a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:33.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4bdfe9640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4a4003ff0 0x7fe4a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4bdfe9640 1 -- 172.21.15.17:0/2759077671 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a4003530 con 0x7fe4a4004d50 2024-03-16T10:10:33.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.318+0000 7fe4bcfe7640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4a4003ff0 0x7fe4a4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:10:33.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.326+0000 7fe4bdfe9640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4a4004d50 0x7fe4a40051d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e8d0 tx=0x7fe4b401c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46cdf94c5e1d4331 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:33.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.326+0000 7fe4affff640 1 -- 172.21.15.17:0/2759077671 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b40ae070 con 0x7fe4a4004d50 2024-03-16T10:10:33.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.326+0000 7fe4affff640 1 -- 172.21.15.17:0/2759077671 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b40a9680 con 0x7fe4a4004d50 2024-03-16T10:10:33.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.326+0000 7fe4affff640 1 -- 172.21.15.17:0/2759077671 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400f040 con 0x7fe4a4004d50 2024-03-16T10:10:33.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.326+0000 7fe4bfa4c640 1 -- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4a4004d50 msgr2=0x7fe4a40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:33.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.326+0000 7fe4bfa4c640 1 --2- 172.21.15.17:0/2759077671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4a4004d50 0x7fe4a40051d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e8d0 tx=0x7fe4b401c8b0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.326+0000 7fe4bdfe9640 1 -- 172.21.15.17:0/2759077671 reap_dead start 2024-03-16T10:10:33.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.326+0000 7fe4bfa4c640 1 -- 172.21.15.17:0/2759077671 shutdown_connections 2024-03-16T10:10:33.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.326+0000 7fe4bfa4c640 1 -- 172.21.15.17:0/2759077671 >> 172.21.15.17:0/2759077671 conn(0x7fe4b806b290 msgr2=0x7fe4b806b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:33.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.326+0000 7fe4bfa4c640 1 -- 172.21.15.17:0/2759077671 shutdown_connections 2024-03-16T10:10:33.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.326+0000 7fe4bfa4c640 1 -- 172.21.15.17:0/2759077671 wait complete. 2024-03-16T10:10:33.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bfa4c640 1 Processor -- start 2024-03-16T10:10:33.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bfa4c640 1 -- start start 2024-03-16T10:10:33.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bfa4c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4a4004d50 0x7fe4b80baf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:33.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bfa4c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4b80ba8c0 0x7fe4b81df9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:33.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bfa4c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b81dfef0 0x7fe4b8134c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bfa4c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8152510 con 0x7fe4b81dfef0 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bfa4c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b80b5f10 con 0x7fe4b80ba8c0 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bfa4c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b8152690 con 0x7fe4a4004d50 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bdfe9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b81dfef0 0x7fe4b8134c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bdfe9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b81dfef0 0x7fe4b8134c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54198/0 (socket says 172.21.15.17:54198) 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bdfe9640 1 -- 172.21.15.17:0/2245980651 learned_addr learned my addr 172.21.15.17:0/2245980651 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bcfe7640 1 --2- 172.21.15.17:0/2245980651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4b80ba8c0 0x7fe4b81df9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bd7e8640 1 --2- 172.21.15.17:0/2245980651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4a4004d50 0x7fe4b80baf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bdfe9640 1 -- 172.21.15.17:0/2245980651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4a4004d50 msgr2=0x7fe4b80baf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bdfe9640 1 --2- 172.21.15.17:0/2245980651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4a4004d50 0x7fe4b80baf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bdfe9640 1 -- 172.21.15.17:0/2245980651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4b80ba8c0 msgr2=0x7fe4b81df9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bdfe9640 1 --2- 172.21.15.17:0/2245980651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4b80ba8c0 0x7fe4b81df9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bdfe9640 1 -- 172.21.15.17:0/2245980651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b81361e0 con 0x7fe4b81dfef0 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bdfe9640 1 --2- 172.21.15.17:0/2245980651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b81dfef0 0x7fe4b8134c70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e8d0 tx=0x7fe4a4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4adffb640 1 -- 172.21.15.17:0/2245980651 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b40ae040 con 0x7fe4b81dfef0 2024-03-16T10:10:33.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4adffb640 1 -- 172.21.15.17:0/2245980651 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b40a9830 con 0x7fe4b81dfef0 2024-03-16T10:10:33.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4adffb640 1 -- 172.21.15.17:0/2245980651 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a800f040 con 0x7fe4b81dfef0 2024-03-16T10:10:33.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bfa4c640 1 -- 172.21.15.17:0/2245980651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b81e0370 con 0x7fe4b81dfef0 2024-03-16T10:10:33.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4bfa4c640 1 -- 172.21.15.17:0/2245980651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b8136450 con 0x7fe4b81dfef0 2024-03-16T10:10:33.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.330+0000 7fe4adffb640 1 -- 172.21.15.17:0/2245980651 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe4b40b1070 con 0x7fe4b81dfef0 2024-03-16T10:10:33.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.334+0000 7fe4adffb640 1 -- 172.21.15.17:0/2245980651 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe4b40b3b80 con 0x7fe4b81dfef0 2024-03-16T10:10:33.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.334+0000 7fe4bfa4c640 1 -- 172.21.15.17:0/2245980651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b806a130 con 0x7fe4b81dfef0 2024-03-16T10:10:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.334+0000 7fe4adffb640 1 -- 172.21.15.17:0/2245980651 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe4b4016600 con 0x7fe4b81dfef0 2024-03-16T10:10:33.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.402+0000 7fe4bfa4c640 1 -- 172.21.15.17:0/2245980651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fe4b813fe90 con 0x7fe4b81dfef0 2024-03-16T10:10:33.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.402+0000 7fe4adffb640 1 -- 172.21.15.17:0/2245980651 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fe4b40b3d80 con 0x7fe4b81dfef0 2024-03-16T10:10:33.406 INFO:teuthology.orchestra.run.smithi017.stderr:adjusted tunables profile to default 2024-03-16T10:10:33.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.406+0000 7fe4bfa4c640 1 -- 172.21.15.17:0/2245980651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b81dfef0 msgr2=0x7fe4b8134c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:33.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.406+0000 7fe4bfa4c640 1 --2- 172.21.15.17:0/2245980651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b81dfef0 0x7fe4b8134c70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e8d0 tx=0x7fe4a4005840 comp rx=0 tx=0).stop 2024-03-16T10:10:33.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.406+0000 7fe4bfa4c640 1 -- 172.21.15.17:0/2245980651 shutdown_connections 2024-03-16T10:10:33.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.406+0000 7fe4bfa4c640 1 --2- 172.21.15.17:0/2245980651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b81dfef0 0x7fe4b8134c70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.406+0000 7fe4bfa4c640 1 --2- 172.21.15.17:0/2245980651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4b80ba8c0 0x7fe4b81df9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.406+0000 7fe4bfa4c640 1 --2- 172.21.15.17:0/2245980651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe4a4004d50 0x7fe4b80baf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.406+0000 7fe4bfa4c640 1 -- 172.21.15.17:0/2245980651 >> 172.21.15.17:0/2245980651 conn(0x7fe4b806b290 msgr2=0x7fe4b8150f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:33.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.406+0000 7fe4bfa4c640 1 -- 172.21.15.17:0/2245980651 shutdown_connections 2024-03-16T10:10:33.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:33.406+0000 7fe4bfa4c640 1 -- 172.21.15.17:0/2245980651 wait complete. 2024-03-16T10:10:33.425 INFO:tasks.ceph:check_enable_crimson: False 2024-03-16T10:10:33.425 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-16T10:10:33.425 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:33.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-16T10:10:33.435 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:33.435 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-16T10:10:33.487 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:33.487 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-16T10:10:33.539 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:10:33.539 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-16T10:10:33.546 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:33.546+0000 7f2adf639180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T10:10:33.591 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:10:33.591 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-16T10:10:33.604 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:10:33.604 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-16T10:10:33.657 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:10:33.657 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-16T10:10:33.709 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-03-16T10:10:33.709 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-16T10:10:33.761 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new dd7d0093-3aa1-4b83-b1e0-b997bae8231f 0 2024-03-16T10:10:33.849 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:33.846+0000 7faa7f5b9180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T10:10:33.920 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c1b9ac640 1 Processor -- start 2024-03-16T10:10:33.921 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c1b9ac640 1 -- start start 2024-03-16T10:10:33.922 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c1b9ac640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9c140aeaf0 0x7f9c14071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:33.922 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c1b9ac640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c14072410 0x7f9c1414e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:33.922 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c1b9ac640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c140aef60 con 0x7f9c140b5740 2024-03-16T10:10:33.922 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c1b9ac640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f9c1406e950 con 0x7f9c140aeaf0 2024-03-16T10:10:33.922 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c1b9ac640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9c140b5b20 con 0x7f9c14072410 2024-03-16T10:10:33.922 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c18f47640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9c140aeaf0 0x7f9c14071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.922 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c19f49640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c14072410 0x7f9c1414e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.922 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c19f49640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c14072410 0x7f9c1414e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59148/0 (socket says 172.21.15.174:59148) 2024-03-16T10:10:33.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c19f49640 1 -- 172.21.15.174:0/739070103 learned_addr learned my addr 172.21.15.174:0/739070103 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-16T10:10:33.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c18f47640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9c140aeaf0 0x7f9c14071ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40642/0 (socket says 172.21.15.174:40642) 2024-03-16T10:10:33.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c19f49640 1 -- 172.21.15.174:0/739070103 >> v2:172.21.15.174:3300/0 conn(0x7f9c140aeaf0 msgr2=0x7f9c14071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:33.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c19f49640 1 --2- 172.21.15.174:0/739070103 >> v2:172.21.15.174:3300/0 conn(0x7f9c140aeaf0 0x7f9c14071ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c19f49640 1 -- 172.21.15.174:0/739070103 >> v1:172.21.15.17:6789/0 conn(0x7f9c140b5740 legacy=0x7f9c1406ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:33.923 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.918+0000 7f9c19f49640 1 -- 172.21.15.174:0/739070103 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c140af720 con 0x7f9c14072410 2024-03-16T10:10:33.925 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.922+0000 7f9c19f49640 1 --2- 172.21.15.174:0/739070103 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c14072410 0x7f9c1414e9b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000bb10 tx=0x7f9c1001ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d6362769a1f2443 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:33.925 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.922+0000 7f9c0bfff640 1 -- 172.21.15.174:0/739070103 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c100b4070 con 0x7f9c14072410 2024-03-16T10:10:33.925 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.922+0000 7f9c0bfff640 1 -- 172.21.15.174:0/739070103 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c100a7a00 con 0x7f9c14072410 2024-03-16T10:10:33.925 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.922+0000 7f9c0bfff640 1 -- 172.21.15.174:0/739070103 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c100a7dd0 con 0x7f9c14072410 2024-03-16T10:10:33.926 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.922+0000 7f9c1b9ac640 1 -- 172.21.15.174:0/739070103 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c14072410 msgr2=0x7f9c1414e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:33.926 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.922+0000 7f9c1b9ac640 1 --2- 172.21.15.174:0/739070103 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c14072410 0x7f9c1414e9b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000bb10 tx=0x7f9c1001ec30 comp rx=0 tx=0).stop 2024-03-16T10:10:33.926 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.922+0000 7f9c1b9ac640 1 -- 172.21.15.174:0/739070103 shutdown_connections 2024-03-16T10:10:33.926 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.922+0000 7f9c1b9ac640 1 --2- 172.21.15.174:0/739070103 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c14072410 0x7f9c1414e9b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.926 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.922+0000 7f9c1b9ac640 1 --2- 172.21.15.174:0/739070103 >> v2:172.21.15.174:3300/0 conn(0x7f9c140aeaf0 0x7f9c14071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.926 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.922+0000 7f9c1b9ac640 1 -- 172.21.15.174:0/739070103 >> 172.21.15.174:0/739070103 conn(0x7f9c140b65d0 msgr2=0x7f9c140b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:33.926 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.922+0000 7f9c1b9ac640 1 -- 172.21.15.174:0/739070103 shutdown_connections 2024-03-16T10:10:33.927 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.922+0000 7f9c1b9ac640 1 -- 172.21.15.174:0/739070103 wait complete. 2024-03-16T10:10:33.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c1b9ac640 1 Processor -- start 2024-03-16T10:10:33.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c1b9ac640 1 -- start start 2024-03-16T10:10:33.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c1b9ac640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c14072410 0x7f9c141e0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:33.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c1b9ac640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c140aeaf0 0x7f9c14143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:33.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c1b9ac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c140b5740 0x7f9c14144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:33.929 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c1b9ac640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9c141477a0 con 0x7f9c14072410 2024-03-16T10:10:33.930 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c1b9ac640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9c14147620 con 0x7f9c140b5740 2024-03-16T10:10:33.930 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c1b9ac640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9c1406baa0 con 0x7f9c140aeaf0 2024-03-16T10:10:33.930 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c19f49640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c140b5740 0x7f9c14144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.930 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c18f47640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c140aeaf0 0x7f9c14143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.930 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c19f49640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c140b5740 0x7f9c14144030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40644/0 (socket says 172.21.15.174:40644) 2024-03-16T10:10:33.930 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c19f49640 1 -- 172.21.15.174:0/1601975926 learned_addr learned my addr 172.21.15.174:0/1601975926 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-16T10:10:33.930 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c19748640 1 --2- 172.21.15.174:0/1601975926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c14072410 0x7f9c141e0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:33.930 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c18f47640 1 -- 172.21.15.174:0/1601975926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c140b5740 msgr2=0x7f9c14144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:33.930 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c18f47640 1 --2- 172.21.15.174:0/1601975926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c140b5740 0x7f9c14144030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.930 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c18f47640 1 -- 172.21.15.174:0/1601975926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c14072410 msgr2=0x7f9c141e0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:33.930 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c18f47640 1 --2- 172.21.15.174:0/1601975926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c14072410 0x7f9c141e0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:33.930 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c18f47640 1 -- 172.21.15.174:0/1601975926 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c141e00d0 con 0x7f9c140aeaf0 2024-03-16T10:10:33.930 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c19f49640 1 --2- 172.21.15.174:0/1601975926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c140b5740 0x7f9c14144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:10:33.930 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c18f47640 1 --2- 172.21.15.174:0/1601975926 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c140aeaf0 0x7f9c14143af0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400e880 tx=0x7f9c0400ed40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:33.931 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c09ffb640 1 -- 172.21.15.174:0/1601975926 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c04017070 con 0x7f9c140aeaf0 2024-03-16T10:10:33.931 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c09ffb640 1 -- 172.21.15.174:0/1601975926 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c0400c940 con 0x7f9c140aeaf0 2024-03-16T10:10:33.931 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c1b9ac640 1 -- 172.21.15.174:0/1601975926 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c14141f20 con 0x7f9c140aeaf0 2024-03-16T10:10:33.931 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c09ffb640 1 -- 172.21.15.174:0/1601975926 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0400cd30 con 0x7f9c140aeaf0 2024-03-16T10:10:33.931 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c1b9ac640 1 -- 172.21.15.174:0/1601975926 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c14147050 con 0x7f9c140aeaf0 2024-03-16T10:10:33.931 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c09ffb640 1 -- 172.21.15.174:0/1601975926 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9c0401a070 con 0x7f9c140aeaf0 2024-03-16T10:10:33.931 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c09ffb640 1 -- 172.21.15.174:0/1601975926 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f9c04013470 con 0x7f9c140aeaf0 2024-03-16T10:10:33.931 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.926+0000 7f9c1b9ac640 1 -- 172.21.15.174:0/1601975926 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c1406a130 con 0x7f9c140aeaf0 2024-03-16T10:10:33.946 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:33.946+0000 7f9c09ffb640 1 -- 172.21.15.174:0/1601975926 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f9c040136c0 con 0x7f9c140aeaf0 2024-03-16T10:10:34.005 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:34.002+0000 7f2adf639180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T10:10:34.016 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.014+0000 7f9c1b9ac640 1 -- 172.21.15.174:0/1601975926 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dd7d0093-3aa1-4b83-b1e0-b997bae8231f", "id": 0} v 0) -- 0x7f9c14071f60 con 0x7f9c140aeaf0 2024-03-16T10:10:34.024 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.022+0000 7f9c09ffb640 1 -- 172.21.15.174:0/1601975926 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd7d0093-3aa1-4b83-b1e0-b997bae8231f", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f9c04013970 con 0x7f9c140aeaf0 2024-03-16T10:10:34.025 INFO:teuthology.orchestra.run.smithi174.stdout:0 2024-03-16T10:10:34.026 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.022+0000 7f9c1b9ac640 1 -- 172.21.15.174:0/1601975926 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c140aeaf0 msgr2=0x7f9c14143af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.026 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.022+0000 7f9c1b9ac640 1 --2- 172.21.15.174:0/1601975926 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c140aeaf0 0x7f9c14143af0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9c0400e880 tx=0x7f9c0400ed40 comp rx=0 tx=0).stop 2024-03-16T10:10:34.026 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.022+0000 7f9c1b9ac640 1 -- 172.21.15.174:0/1601975926 shutdown_connections 2024-03-16T10:10:34.026 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.022+0000 7f9c1b9ac640 1 --2- 172.21.15.174:0/1601975926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c140b5740 0x7f9c14144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.026 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.022+0000 7f9c1b9ac640 1 --2- 172.21.15.174:0/1601975926 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9c140aeaf0 0x7f9c14143af0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.027 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.022+0000 7f9c1b9ac640 1 --2- 172.21.15.174:0/1601975926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c14072410 0x7f9c141e0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.027 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.022+0000 7f9c1b9ac640 1 -- 172.21.15.174:0/1601975926 >> 172.21.15.174:0/1601975926 conn(0x7f9c140b65d0 msgr2=0x7f9c140741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:34.027 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.022+0000 7f9c1b9ac640 1 -- 172.21.15.174:0/1601975926 shutdown_connections 2024-03-16T10:10:34.027 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.022+0000 7f9c1b9ac640 1 -- 172.21.15.174:0/1601975926 wait complete. 2024-03-16T10:10:34.041 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new d043363d-4176-4bcd-a4f4-aadcd5c14d97 1 2024-03-16T10:10:34.161 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.158+0000 7f8177ac6640 1 Processor -- start 2024-03-16T10:10:34.161 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.158+0000 7f8177ac6640 1 -- start start 2024-03-16T10:10:34.163 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.158+0000 7f8177ac6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8170073070 0x7f81701410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.163 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.158+0000 7f8177ac6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f817006ce50 con 0x7f81700b63a0 2024-03-16T10:10:34.163 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.158+0000 7f8177ac6640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81700b6780 con 0x7f81700b7b70 2024-03-16T10:10:34.163 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.158+0000 7f8177ac6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f81700b6cc0 con 0x7f8170073070 2024-03-16T10:10:34.163 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.158+0000 7f8176063640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f81700b7b70 0x7f8170151d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:55312/0 (socket says 172.21.15.174:55312) 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.158+0000 7f8176063640 1 -- 172.21.15.174:0/2510067969 learned_addr learned my addr 172.21.15.174:0/2510067969 (peer_addr_for_me v1:172.21.15.174:0/0) 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.158+0000 7f8175061640 1 --2- 172.21.15.174:0/2510067969 >> v2:172.21.15.17:3300/0 conn(0x7f8170073070 0x7f81701410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.158+0000 7f8174860640 1 -- 172.21.15.174:0/2510067969 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708250213 0 0) 0x7f81700b6780 con 0x7f81700b7b70 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.158+0000 7f8174860640 1 -- 172.21.15.174:0/2510067969 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f815c003530 con 0x7f81700b7b70 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.158+0000 7f8174860640 1 -- 172.21.15.174:0/2510067969 <== mon.1 v1:172.21.15.174:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2610669645 0 0) 0x7f815c003530 con 0x7f81700b7b70 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.158+0000 7f8174860640 1 -- 172.21.15.174:0/2510067969 >> v2:172.21.15.17:3300/0 conn(0x7f8170073070 msgr2=0x7f81701410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.158+0000 7f8174860640 1 --2- 172.21.15.174:0/2510067969 >> v2:172.21.15.17:3300/0 conn(0x7f8170073070 0x7f81701410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8174860640 1 -- 172.21.15.174:0/2510067969 >> v1:172.21.15.17:6789/0 conn(0x7f81700b63a0 legacy=0x7f817006d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8174860640 1 -- 172.21.15.174:0/2510067969 --> v1:172.21.15.174:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81700b6fa0 con 0x7f81700b7b70 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8174860640 1 -- 172.21.15.174:0/2510067969 <== mon.1 v1:172.21.15.174:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f8168001a00 con 0x7f81700b7b70 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8174860640 1 -- 172.21.15.174:0/2510067969 >> v1:172.21.15.174:6789/0 conn(0x7f81700b7b70 legacy=0x7f8170151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8174860640 1 --2- 172.21.15.174:0/2510067969 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f815c0039c0 0x7f815c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8174860640 1 --2- 172.21.15.174:0/2510067969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815c004810 0x7f815c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8174860640 1 --2- 172.21.15.174:0/2510067969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f815c0051d0 0x7f815c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.164 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8174860640 1 -- 172.21.15.174:0/2510067969 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f81700b6cc0 con 0x7f815c0051d0 2024-03-16T10:10:34.165 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8174860640 1 -- 172.21.15.174:0/2510067969 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f815c00c170 con 0x7f815c004810 2024-03-16T10:10:34.165 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8174860640 1 -- 172.21.15.174:0/2510067969 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f815c00c370 con 0x7f815c0039c0 2024-03-16T10:10:34.165 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8175061640 1 --2- 172.21.15.174:0/2510067969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815c004810 0x7f815c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.165 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8175862640 1 --2- 172.21.15.174:0/2510067969 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f815c0039c0 0x7f815c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.165 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8176063640 1 --2- 172.21.15.174:0/2510067969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f815c0051d0 0x7f815c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.165 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8175061640 1 -- 172.21.15.174:0/2510067969 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f815c0039c0 msgr2=0x7f815c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:34.165 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8175061640 1 --2- 172.21.15.174:0/2510067969 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f815c0039c0 0x7f815c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.165 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8175061640 1 -- 172.21.15.174:0/2510067969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f815c0051d0 msgr2=0x7f815c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:34.165 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8175061640 1 --2- 172.21.15.174:0/2510067969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f815c0051d0 0x7f815c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.165 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8175061640 1 -- 172.21.15.174:0/2510067969 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c003530 con 0x7f815c004810 2024-03-16T10:10:34.165 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8176063640 1 --2- 172.21.15.174:0/2510067969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f815c0051d0 0x7f815c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:10:34.165 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8176063640 1 -- 172.21.15.174:0/2510067969 reap_dead start 2024-03-16T10:10:34.167 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8175061640 1 --2- 172.21.15.174:0/2510067969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815c004810 0x7f815c004c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8160009ae0 tx=0x7f816001ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d584c6d42a23a26f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:34.167 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8174860640 1 -- 172.21.15.174:0/2510067969 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81600b2070 con 0x7f815c004810 2024-03-16T10:10:34.167 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8174860640 1 -- 172.21.15.174:0/2510067969 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81600a5660 con 0x7f815c004810 2024-03-16T10:10:34.167 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8174860640 1 -- 172.21.15.174:0/2510067969 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81600096b0 con 0x7f815c004810 2024-03-16T10:10:34.167 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8177ac6640 1 -- 172.21.15.174:0/2510067969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815c004810 msgr2=0x7f815c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.167 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8177ac6640 1 --2- 172.21.15.174:0/2510067969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815c004810 0x7f815c004c90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8160009ae0 tx=0x7f816001ca70 comp rx=0 tx=0).stop 2024-03-16T10:10:34.167 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8177ac6640 1 -- 172.21.15.174:0/2510067969 shutdown_connections 2024-03-16T10:10:34.167 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8177ac6640 1 --2- 172.21.15.174:0/2510067969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815c004810 0x7f815c004c90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.167 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8177ac6640 1 -- 172.21.15.174:0/2510067969 >> 172.21.15.174:0/2510067969 conn(0x7f817006b290 msgr2=0x7f817006b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:34.168 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8177ac6640 1 -- 172.21.15.174:0/2510067969 shutdown_connections 2024-03-16T10:10:34.168 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.162+0000 7f8177ac6640 1 -- 172.21.15.174:0/2510067969 wait complete. 2024-03-16T10:10:34.168 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8177ac6640 1 Processor -- start 2024-03-16T10:10:34.168 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8177ac6640 1 -- start start 2024-03-16T10:10:34.170 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8177ac6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815c004810 0x7f81700b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.170 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8177ac6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81700b3dd0 0x7f8170144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.170 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8177ac6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f81701e0600 0x7f81701e0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.170 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8177ac6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f81700b5d00 con 0x7f81700b3dd0 2024-03-16T10:10:34.170 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8177ac6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81700b69c0 con 0x7f815c004810 2024-03-16T10:10:34.170 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8177ac6640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f81700b5e80 con 0x7f81701e0600 2024-03-16T10:10:34.170 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8175862640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815c004810 0x7f81700b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.170 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8175862640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815c004810 0x7f81700b26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40660/0 (socket says 172.21.15.174:40660) 2024-03-16T10:10:34.170 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8175862640 1 -- 172.21.15.174:0/913231148 learned_addr learned my addr 172.21.15.174:0/913231148 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-16T10:10:34.170 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8176063640 1 --2- 172.21.15.174:0/913231148 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f81701e0600 0x7f81701e0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8175061640 1 --2- 172.21.15.174:0/913231148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81700b3dd0 0x7f8170144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8175862640 1 -- 172.21.15.174:0/913231148 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f81701e0600 msgr2=0x7f81701e0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8175862640 1 --2- 172.21.15.174:0/913231148 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f81701e0600 0x7f81701e0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8175862640 1 -- 172.21.15.174:0/913231148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81700b3dd0 msgr2=0x7f8170144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8175862640 1 --2- 172.21.15.174:0/913231148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81700b3dd0 0x7f8170144240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8175862640 1 -- 172.21.15.174:0/913231148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8170144780 con 0x7f815c004810 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8175061640 1 --2- 172.21.15.174:0/913231148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81700b3dd0 0x7f8170144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8175862640 1 --2- 172.21.15.174:0/913231148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815c004810 0x7f81700b26c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f816c00f820 tx=0x7f816c00fc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f81667fc640 1 -- 172.21.15.174:0/913231148 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c0050d0 con 0x7f815c004810 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f81667fc640 1 -- 172.21.15.174:0/913231148 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f816c018070 con 0x7f815c004810 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8177ac6640 1 -- 172.21.15.174:0/913231148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81700b4320 con 0x7f815c004810 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f81667fc640 1 -- 172.21.15.174:0/913231148 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816c00d730 con 0x7f815c004810 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8177ac6640 1 -- 172.21.15.174:0/913231148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81700b1fc0 con 0x7f815c004810 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f81667fc640 1 -- 172.21.15.174:0/913231148 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f816c00d8d0 con 0x7f815c004810 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f81667fc640 1 -- 172.21.15.174:0/913231148 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f816c01d810 con 0x7f815c004810 2024-03-16T10:10:34.171 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.166+0000 7f8177ac6640 1 -- 172.21.15.174:0/913231148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f817006a130 con 0x7f815c004810 2024-03-16T10:10:34.172 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.170+0000 7f81667fc640 1 -- 172.21.15.174:0/913231148 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f816c00dd50 con 0x7f815c004810 2024-03-16T10:10:34.219 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:34.218+0000 7f2adf639180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T10:10:34.240 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.238+0000 7f8177ac6640 1 -- 172.21.15.174:0/913231148 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d043363d-4176-4bcd-a4f4-aadcd5c14d97", "id": 1} v 0) -- 0x7f81701459b0 con 0x7f815c004810 2024-03-16T10:10:34.249 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.246+0000 7f81667fc640 1 -- 172.21.15.174:0/913231148 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d043363d-4176-4bcd-a4f4-aadcd5c14d97", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f816c003600 con 0x7f815c004810 2024-03-16T10:10:34.249 INFO:teuthology.orchestra.run.smithi174.stdout:1 2024-03-16T10:10:34.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.246+0000 7f8177ac6640 1 -- 172.21.15.174:0/913231148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815c004810 msgr2=0x7f81700b26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.246+0000 7f8177ac6640 1 --2- 172.21.15.174:0/913231148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815c004810 0x7f81700b26c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f816c00f820 tx=0x7f816c00fc20 comp rx=0 tx=0).stop 2024-03-16T10:10:34.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.246+0000 7f8177ac6640 1 -- 172.21.15.174:0/913231148 shutdown_connections 2024-03-16T10:10:34.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.246+0000 7f8177ac6640 1 --2- 172.21.15.174:0/913231148 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f81701e0600 0x7f81701e0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.246+0000 7f8177ac6640 1 --2- 172.21.15.174:0/913231148 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81700b3dd0 0x7f8170144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.246+0000 7f8177ac6640 1 --2- 172.21.15.174:0/913231148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f815c004810 0x7f81700b26c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.251 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.246+0000 7f8177ac6640 1 -- 172.21.15.174:0/913231148 >> 172.21.15.174:0/913231148 conn(0x7f817006b290 msgr2=0x7f81701d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:34.251 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.246+0000 7f8177ac6640 1 -- 172.21.15.174:0/913231148 shutdown_connections 2024-03-16T10:10:34.251 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.246+0000 7f8177ac6640 1 -- 172.21.15.174:0/913231148 wait complete. 2024-03-16T10:10:34.265 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 48afbf8a-dcdd-4301-8a7e-ce26fd327df9 2 2024-03-16T10:10:34.332 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:34.330+0000 7faa7f5b9180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T10:10:34.385 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.382+0000 7f324b00b640 1 Processor -- start 2024-03-16T10:10:34.385 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.382+0000 7f324b00b640 1 -- start start 2024-03-16T10:10:34.388 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.382+0000 7f324b00b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f32440b44b0 0x7f32440ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.388 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.382+0000 7f324b00b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3244143f20 con 0x7f324413deb0 2024-03-16T10:10:34.388 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.382+0000 7f324b00b640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3244144200 con 0x7f324413dad0 2024-03-16T10:10:34.388 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.382+0000 7f324b00b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f32441444e0 con 0x7f32440b44b0 2024-03-16T10:10:34.388 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.382+0000 7f3248da7640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f324413dad0 0x7f32440b3da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:55318/0 (socket says 172.21.15.174:55318) 2024-03-16T10:10:34.388 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.382+0000 7f3248da7640 1 -- 172.21.15.174:0/1064473283 learned_addr learned my addr 172.21.15.174:0/1064473283 (peer_addr_for_me v1:172.21.15.174:0/0) 2024-03-16T10:10:34.388 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f3243fff640 1 --2- 172.21.15.174:0/1064473283 >> v2:172.21.15.17:3300/0 conn(0x7f32440b44b0 0x7f32440ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 -- 172.21.15.174:0/1064473283 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1592838963 0 0) 0x7f3244144200 con 0x7f324413dad0 2024-03-16T10:10:34.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 -- 172.21.15.174:0/1064473283 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3234003450 con 0x7f324413dad0 2024-03-16T10:10:34.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 -- 172.21.15.174:0/1064473283 <== mon.1 v1:172.21.15.174:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3918481283 0 0) 0x7f3234003450 con 0x7f324413dad0 2024-03-16T10:10:34.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 -- 172.21.15.174:0/1064473283 >> v2:172.21.15.17:3300/0 conn(0x7f32440b44b0 msgr2=0x7f32440ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:34.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 --2- 172.21.15.174:0/1064473283 >> v2:172.21.15.17:3300/0 conn(0x7f32440b44b0 0x7f32440ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 -- 172.21.15.174:0/1064473283 >> v1:172.21.15.17:6789/0 conn(0x7f324413deb0 legacy=0x7f32440aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 -- 172.21.15.174:0/1064473283 --> v1:172.21.15.174:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324406e270 con 0x7f324413dad0 2024-03-16T10:10:34.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 -- 172.21.15.174:0/1064473283 <== mon.1 v1:172.21.15.174:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f3238001680 con 0x7f324413dad0 2024-03-16T10:10:34.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 -- 172.21.15.174:0/1064473283 >> v1:172.21.15.174:6789/0 conn(0x7f324413dad0 legacy=0x7f32440b3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 --2- 172.21.15.174:0/1064473283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3234003ab0 0x7f3234003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 --2- 172.21.15.174:0/1064473283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3234008860 0x7f323400acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 --2- 172.21.15.174:0/1064473283 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f323400b570 0x7f323400da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 -- 172.21.15.174:0/1064473283 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f32441444e0 con 0x7f3234008860 2024-03-16T10:10:34.389 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 -- 172.21.15.174:0/1064473283 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f323400e310 con 0x7f3234003ab0 2024-03-16T10:10:34.390 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 -- 172.21.15.174:0/1064473283 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f323400e510 con 0x7f323400b570 2024-03-16T10:10:34.390 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f3248da7640 1 --2- 172.21.15.174:0/1064473283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3234003ab0 0x7f3234003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.390 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32495a8640 1 --2- 172.21.15.174:0/1064473283 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f323400b570 0x7f323400da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.390 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f3243fff640 1 --2- 172.21.15.174:0/1064473283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3234008860 0x7f323400acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.390 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f3248da7640 1 -- 172.21.15.174:0/1064473283 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f323400b570 msgr2=0x7f323400da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:34.390 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f3248da7640 1 --2- 172.21.15.174:0/1064473283 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f323400b570 0x7f323400da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.390 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f3248da7640 1 -- 172.21.15.174:0/1064473283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3234008860 msgr2=0x7f323400acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:34.390 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f3248da7640 1 --2- 172.21.15.174:0/1064473283 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3234008860 0x7f323400acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.390 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f3248da7640 1 -- 172.21.15.174:0/1064473283 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3234003450 con 0x7f3234003ab0 2024-03-16T10:10:34.390 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32495a8640 1 --2- 172.21.15.174:0/1064473283 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f323400b570 0x7f323400da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:10:34.390 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32495a8640 1 -- 172.21.15.174:0/1064473283 reap_dead start 2024-03-16T10:10:34.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f3248da7640 1 --2- 172.21.15.174:0/1064473283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3234003ab0 0x7f3234003ed0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f323800c210 tx=0x7f323801cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4163da45ff9033e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:34.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 -- 172.21.15.174:0/1064473283 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238007d40 con 0x7f3234003ab0 2024-03-16T10:10:34.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 -- 172.21.15.174:0/1064473283 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32380a9070 con 0x7f3234003ab0 2024-03-16T10:10:34.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f32437fe640 1 -- 172.21.15.174:0/1064473283 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3238005d80 con 0x7f3234003ab0 2024-03-16T10:10:34.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f324b00b640 1 -- 172.21.15.174:0/1064473283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3234003ab0 msgr2=0x7f3234003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f324b00b640 1 --2- 172.21.15.174:0/1064473283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3234003ab0 0x7f3234003ed0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f323800c210 tx=0x7f323801cc60 comp rx=0 tx=0).stop 2024-03-16T10:10:34.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f324b00b640 1 -- 172.21.15.174:0/1064473283 shutdown_connections 2024-03-16T10:10:34.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f324b00b640 1 --2- 172.21.15.174:0/1064473283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3234003ab0 0x7f3234003ed0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.386+0000 7f324b00b640 1 -- 172.21.15.174:0/1064473283 >> 172.21.15.174:0/1064473283 conn(0x7f32440b5ea0 msgr2=0x7f32440b6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:34.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f324b00b640 1 -- 172.21.15.174:0/1064473283 shutdown_connections 2024-03-16T10:10:34.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f324b00b640 1 -- 172.21.15.174:0/1064473283 wait complete. 2024-03-16T10:10:34.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f324b00b640 1 Processor -- start 2024-03-16T10:10:34.394 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f324b00b640 1 -- start start 2024-03-16T10:10:34.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f324b00b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3234003ab0 0x7f32441dd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f324b00b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3244072ca0 0x7f324409e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f324b00b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32441ddb30 0x7f324409e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f324b00b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f32440aed30 con 0x7f3234003ab0 2024-03-16T10:10:34.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f324b00b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32440aebb0 con 0x7f32441ddb30 2024-03-16T10:10:34.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f324b00b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f32440aeeb0 con 0x7f3244072ca0 2024-03-16T10:10:34.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f32495a8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32441ddb30 0x7f324409e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f3243fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3244072ca0 0x7f324409e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f32495a8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32441ddb30 0x7f324409e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40674/0 (socket says 172.21.15.174:40674) 2024-03-16T10:10:34.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f3243fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3244072ca0 0x7f324409e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59190/0 (socket says 172.21.15.174:59190) 2024-03-16T10:10:34.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f32495a8640 1 -- 172.21.15.174:0/3655884366 learned_addr learned my addr 172.21.15.174:0/3655884366 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-16T10:10:34.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f3248da7640 1 --2- 172.21.15.174:0/3655884366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3234003ab0 0x7f32441dd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.395 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f32495a8640 1 -- 172.21.15.174:0/3655884366 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3244072ca0 msgr2=0x7f324409e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f32495a8640 1 --2- 172.21.15.174:0/3655884366 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3244072ca0 0x7f324409e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f32495a8640 1 -- 172.21.15.174:0/3655884366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3234003ab0 msgr2=0x7f32441dd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f32495a8640 1 --2- 172.21.15.174:0/3655884366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3234003ab0 0x7f32441dd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f32495a8640 1 -- 172.21.15.174:0/3655884366 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3244073120 con 0x7f32441ddb30 2024-03-16T10:10:34.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f3243fff640 1 --2- 172.21.15.174:0/3655884366 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3244072ca0 0x7f324409e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:10:34.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f32495a8640 1 --2- 172.21.15.174:0/3655884366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32441ddb30 0x7f324409e9a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f32440b6c40 tx=0x7f323c00feb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:34.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f32417fa640 1 -- 172.21.15.174:0/3655884366 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c001a90 con 0x7f32441ddb30 2024-03-16T10:10:34.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f32417fa640 1 -- 172.21.15.174:0/3655884366 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323c018070 con 0x7f32441ddb30 2024-03-16T10:10:34.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f324b00b640 1 -- 172.21.15.174:0/3655884366 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3244073400 con 0x7f32441ddb30 2024-03-16T10:10:34.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f32417fa640 1 -- 172.21.15.174:0/3655884366 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c003dd0 con 0x7f32441ddb30 2024-03-16T10:10:34.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f324b00b640 1 -- 172.21.15.174:0/3655884366 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3244070f50 con 0x7f32441ddb30 2024-03-16T10:10:34.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f32417fa640 1 -- 172.21.15.174:0/3655884366 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f323c0050d0 con 0x7f32441ddb30 2024-03-16T10:10:34.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f32417fa640 1 -- 172.21.15.174:0/3655884366 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f323c00dd80 con 0x7f32441ddb30 2024-03-16T10:10:34.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.390+0000 7f324b00b640 1 -- 172.21.15.174:0/3655884366 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324406a130 con 0x7f32441ddb30 2024-03-16T10:10:34.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.394+0000 7f32417fa640 1 -- 172.21.15.174:0/3655884366 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f323c0055a0 con 0x7f32441ddb30 2024-03-16T10:10:34.463 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.462+0000 7f324b00b640 1 -- 172.21.15.174:0/3655884366 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd new", "uuid": "48afbf8a-dcdd-4301-8a7e-ce26fd327df9", "id": 2} v 0) -- 0x7f32440725a0 con 0x7f32441ddb30 2024-03-16T10:10:34.471 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.470+0000 7f32417fa640 1 -- 172.21.15.174:0/3655884366 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "48afbf8a-dcdd-4301-8a7e-ce26fd327df9", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f323c00d740 con 0x7f32441ddb30 2024-03-16T10:10:34.472 INFO:teuthology.orchestra.run.smithi174.stdout:2 2024-03-16T10:10:34.473 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.470+0000 7f324b00b640 1 -- 172.21.15.174:0/3655884366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32441ddb30 msgr2=0x7f324409e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.473 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.470+0000 7f324b00b640 1 --2- 172.21.15.174:0/3655884366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32441ddb30 0x7f324409e9a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f32440b6c40 tx=0x7f323c00feb0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.473 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.470+0000 7f324b00b640 1 -- 172.21.15.174:0/3655884366 shutdown_connections 2024-03-16T10:10:34.473 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.470+0000 7f324b00b640 1 --2- 172.21.15.174:0/3655884366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32441ddb30 0x7f324409e9a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.474 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.470+0000 7f324b00b640 1 --2- 172.21.15.174:0/3655884366 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3244072ca0 0x7f324409e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.474 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.470+0000 7f324b00b640 1 --2- 172.21.15.174:0/3655884366 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3234003ab0 0x7f32441dd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.474 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.470+0000 7f324b00b640 1 -- 172.21.15.174:0/3655884366 >> 172.21.15.174:0/3655884366 conn(0x7f32440b5ea0 msgr2=0x7f32440b6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:34.474 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.470+0000 7f324b00b640 1 -- 172.21.15.174:0/3655884366 shutdown_connections 2024-03-16T10:10:34.474 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.470+0000 7f324b00b640 1 -- 172.21.15.174:0/3655884366 wait complete. 2024-03-16T10:10:34.488 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 4f84ab8d-fb79-41ba-ae35-09aeba2105f4 3 2024-03-16T10:10:34.524 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:34.522+0000 7faa7f5b9180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T10:10:34.605 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.602+0000 7fea0f536640 1 Processor -- start 2024-03-16T10:10:34.605 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.602+0000 7fea0f536640 1 -- start start 2024-03-16T10:10:34.607 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.602+0000 7fea0f536640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fea08067220 0x7fea081478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.607 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.602+0000 7fea0f536640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fea080b4e50 0x7fea08147e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.607 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.602+0000 7fea0f536640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea080aeee0 con 0x7fea081414d0 2024-03-16T10:10:34.607 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.602+0000 7fea0f536640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fea080af150 con 0x7fea080b4e50 2024-03-16T10:10:34.607 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.602+0000 7fea0f536640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fea08067600 con 0x7fea08067220 2024-03-16T10:10:34.607 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.602+0000 7fea0d2d2640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fea08067220 0x7fea081478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.607 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.602+0000 7fea0dad3640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fea081414d0 0x7fea081418e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:55320/0 (socket says 172.21.15.174:55320) 2024-03-16T10:10:34.607 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.602+0000 7fea0dad3640 1 -- 172.21.15.174:0/2537611774 learned_addr learned my addr 172.21.15.174:0/2537611774 (peer_addr_for_me v1:172.21.15.174:0/0) 2024-03-16T10:10:34.607 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fea0cad1640 1 --2- 172.21.15.174:0/2537611774 >> v2:172.21.15.17:3300/0 conn(0x7fea080b4e50 0x7fea08147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.608 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fea0d2d2640 1 -- 172.21.15.174:0/2537611774 >> v2:172.21.15.17:3300/0 conn(0x7fea080b4e50 msgr2=0x7fea08147e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:34.608 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fea0d2d2640 1 --2- 172.21.15.174:0/2537611774 >> v2:172.21.15.17:3300/0 conn(0x7fea080b4e50 0x7fea08147e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.608 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fea0d2d2640 1 -- 172.21.15.174:0/2537611774 >> v1:172.21.15.174:6789/0 conn(0x7fea081414d0 legacy=0x7fea081418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.608 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fea0d2d2640 1 -- 172.21.15.174:0/2537611774 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea080b5400 con 0x7fea08067220 2024-03-16T10:10:34.609 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fea0d2d2640 1 --2- 172.21.15.174:0/2537611774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea08067220 0x7fea081478d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fea0806c2e0 tx=0x7fea0001c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6ccf288975db3d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:34.609 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fe9fffff640 1 -- 172.21.15.174:0/2537611774 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea000b2070 con 0x7fea08067220 2024-03-16T10:10:34.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fe9fffff640 1 -- 172.21.15.174:0/2537611774 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea000a5960 con 0x7fea08067220 2024-03-16T10:10:34.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fe9fffff640 1 -- 172.21.15.174:0/2537611774 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea000a5d30 con 0x7fea08067220 2024-03-16T10:10:34.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fea0f536640 1 -- 172.21.15.174:0/2537611774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea08067220 msgr2=0x7fea081478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fea0f536640 1 --2- 172.21.15.174:0/2537611774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea08067220 0x7fea081478d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fea0806c2e0 tx=0x7fea0001c8a0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fea0f536640 1 -- 172.21.15.174:0/2537611774 shutdown_connections 2024-03-16T10:10:34.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fea0f536640 1 --2- 172.21.15.174:0/2537611774 >> v2:172.21.15.17:3300/0 conn(0x7fea080b4e50 0x7fea08147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fea0f536640 1 --2- 172.21.15.174:0/2537611774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea08067220 0x7fea081478d0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fea0f536640 1 -- 172.21.15.174:0/2537611774 >> 172.21.15.174:0/2537611774 conn(0x7fea0806bd70 msgr2=0x7fea080b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:34.610 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fea0f536640 1 -- 172.21.15.174:0/2537611774 shutdown_connections 2024-03-16T10:10:34.611 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.606+0000 7fea0f536640 1 -- 172.21.15.174:0/2537611774 wait complete. 2024-03-16T10:10:34.613 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0f536640 1 Processor -- start 2024-03-16T10:10:34.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0f536640 1 -- start start 2024-03-16T10:10:34.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0f536640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea08067220 0x7fea08146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0f536640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea080b4e50 0x7fea080b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0f536640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea081414d0 0x7fea080b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0f536640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fea08151460 con 0x7fea080b4e50 2024-03-16T10:10:34.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0f536640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea081512e0 con 0x7fea081414d0 2024-03-16T10:10:34.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0f536640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fea081515e0 con 0x7fea08067220 2024-03-16T10:10:34.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0d2d2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea08067220 0x7fea08146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.614 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0dad3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea081414d0 0x7fea080b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0d2d2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea08067220 0x7fea08146be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59200/0 (socket says 172.21.15.174:59200) 2024-03-16T10:10:34.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0d2d2640 1 -- 172.21.15.174:0/1643039771 learned_addr learned my addr 172.21.15.174:0/1643039771 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-16T10:10:34.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0cad1640 1 --2- 172.21.15.174:0/1643039771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea080b4e50 0x7fea080b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0d2d2640 1 -- 172.21.15.174:0/1643039771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea081414d0 msgr2=0x7fea080b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0d2d2640 1 --2- 172.21.15.174:0/1643039771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea081414d0 0x7fea080b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0d2d2640 1 -- 172.21.15.174:0/1643039771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea080b4e50 msgr2=0x7fea080b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0d2d2640 1 --2- 172.21.15.174:0/1643039771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea080b4e50 0x7fea080b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0d2d2640 1 -- 172.21.15.174:0/1643039771 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea080b1310 con 0x7fea08067220 2024-03-16T10:10:34.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0d2d2640 1 --2- 172.21.15.174:0/1643039771 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea08067220 0x7fea08146be0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea000194b0 tx=0x7fea00019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:34.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fe9fdffb640 1 -- 172.21.15.174:0/1643039771 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea000b2050 con 0x7fea08067220 2024-03-16T10:10:34.615 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fe9fdffb640 1 -- 172.21.15.174:0/1643039771 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea000be070 con 0x7fea08067220 2024-03-16T10:10:34.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0f536640 1 -- 172.21.15.174:0/1643039771 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea081e3520 con 0x7fea08067220 2024-03-16T10:10:34.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fe9fdffb640 1 -- 172.21.15.174:0/1643039771 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea000ada70 con 0x7fea08067220 2024-03-16T10:10:34.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0f536640 1 -- 172.21.15.174:0/1643039771 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fea0809e0a0 con 0x7fea08067220 2024-03-16T10:10:34.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fe9fdffb640 1 -- 172.21.15.174:0/1643039771 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fea000adc10 con 0x7fea08067220 2024-03-16T10:10:34.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fe9fdffb640 1 -- 172.21.15.174:0/1643039771 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fea000c3d20 con 0x7fea08067220 2024-03-16T10:10:34.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.610+0000 7fea0f536640 1 -- 172.21.15.174:0/1643039771 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea0806a130 con 0x7fea08067220 2024-03-16T10:10:34.616 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.614+0000 7fe9fdffb640 1 -- 172.21.15.174:0/1643039771 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fea00016520 con 0x7fea08067220 2024-03-16T10:10:34.683 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.682+0000 7fea0f536640 1 -- 172.21.15.174:0/1643039771 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4f84ab8d-fb79-41ba-ae35-09aeba2105f4", "id": 3} v 0) -- 0x7fea080aeb90 con 0x7fea08067220 2024-03-16T10:10:34.688 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.686+0000 7fe9fdffb640 1 -- 172.21.15.174:0/1643039771 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4f84ab8d-fb79-41ba-ae35-09aeba2105f4", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fea000b9450 con 0x7fea08067220 2024-03-16T10:10:34.688 INFO:teuthology.orchestra.run.smithi174.stdout:3 2024-03-16T10:10:34.689 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.686+0000 7fea0f536640 1 -- 172.21.15.174:0/1643039771 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea08067220 msgr2=0x7fea08146be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.686+0000 7fea0f536640 1 --2- 172.21.15.174:0/1643039771 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea08067220 0x7fea08146be0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fea000194b0 tx=0x7fea00019640 comp rx=0 tx=0).stop 2024-03-16T10:10:34.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.686+0000 7fea0f536640 1 -- 172.21.15.174:0/1643039771 shutdown_connections 2024-03-16T10:10:34.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.686+0000 7fea0f536640 1 --2- 172.21.15.174:0/1643039771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea081414d0 0x7fea080b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.686+0000 7fea0f536640 1 --2- 172.21.15.174:0/1643039771 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fea080b4e50 0x7fea080b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.686+0000 7fea0f536640 1 --2- 172.21.15.174:0/1643039771 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fea08067220 0x7fea08146be0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.686+0000 7fea0f536640 1 -- 172.21.15.174:0/1643039771 >> 172.21.15.174:0/1643039771 conn(0x7fea0806bd70 msgr2=0x7fea0809b520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:34.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.686+0000 7fea0f536640 1 -- 172.21.15.174:0/1643039771 shutdown_connections 2024-03-16T10:10:34.690 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.686+0000 7fea0f536640 1 -- 172.21.15.174:0/1643039771 wait complete. 2024-03-16T10:10:34.705 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new ddb163f0-cce0-42a6-8d92-644babc32d58 4 2024-03-16T10:10:34.791 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:34.790+0000 7f2adf639180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T10:10:34.817 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.814+0000 7f83f0511640 1 Processor -- start 2024-03-16T10:10:34.817 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.814+0000 7f83f0511640 1 -- start start 2024-03-16T10:10:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.814+0000 7f83f0511640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f83e80b63a0 0x7f83e806d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.814+0000 7f83f0511640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f83e8073070 0x7f83e81410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.814+0000 7f83f0511640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e80b7b70 0x7f83e8141630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.814+0000 7f83f0511640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f83e806ce50 con 0x7f83e80b63a0 2024-03-16T10:10:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.814+0000 7f83f0511640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f83e80b6780 con 0x7f83e8073070 2024-03-16T10:10:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.814+0000 7f83f0511640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f83e80b6cc0 con 0x7f83e80b7b70 2024-03-16T10:10:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.814+0000 7f83edaac640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f83e8073070 0x7f83e81410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.814+0000 7f83eeaae640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e80b7b70 0x7f83e8141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83edaac640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f83e8073070 0x7f83e81410f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40708/0 (socket says 172.21.15.174:40708) 2024-03-16T10:10:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83edaac640 1 -- 172.21.15.174:0/1334092108 learned_addr learned my addr 172.21.15.174:0/1334092108 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-16T10:10:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83eeaae640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e80b7b70 0x7f83e8141630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59206/0 (socket says 172.21.15.174:59206) 2024-03-16T10:10:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83ee2ad640 1 --2- 172.21.15.174:0/1334092108 >> v2:172.21.15.17:3300/0 conn(0x7f83e80b63a0 0x7f83e806d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.819 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83eeaae640 1 -- 172.21.15.174:0/1334092108 >> v2:172.21.15.174:3300/0 conn(0x7f83e8073070 msgr2=0x7f83e81410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:34.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83eeaae640 1 --2- 172.21.15.174:0/1334092108 >> v2:172.21.15.174:3300/0 conn(0x7f83e8073070 0x7f83e81410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83eeaae640 1 -- 172.21.15.174:0/1334092108 >> v2:172.21.15.17:3300/0 conn(0x7f83e80b63a0 msgr2=0x7f83e806d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:34.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83eeaae640 1 --2- 172.21.15.174:0/1334092108 >> v2:172.21.15.17:3300/0 conn(0x7f83e80b63a0 0x7f83e806d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83eeaae640 1 -- 172.21.15.174:0/1334092108 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e80b6fa0 con 0x7f83e80b7b70 2024-03-16T10:10:34.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83edaac640 1 --2- 172.21.15.174:0/1334092108 >> v2:172.21.15.174:3300/0 conn(0x7f83e8073070 0x7f83e81410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:10:34.820 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83ee2ad640 1 --2- 172.21.15.174:0/1334092108 >> v2:172.21.15.17:3300/0 conn(0x7f83e80b63a0 0x7f83e806d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:10:34.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83eeaae640 1 --2- 172.21.15.174:0/1334092108 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e80b7b70 0x7f83e8141630 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f83e80b8370 tx=0x7f83e001c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c8d0edf0f631a75 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:34.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83ed2ab640 1 -- 172.21.15.174:0/1334092108 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e00b2070 con 0x7f83e80b7b70 2024-03-16T10:10:34.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83ed2ab640 1 -- 172.21.15.174:0/1334092108 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e00a58c0 con 0x7f83e80b7b70 2024-03-16T10:10:34.821 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83ed2ab640 1 -- 172.21.15.174:0/1334092108 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e00a5c90 con 0x7f83e80b7b70 2024-03-16T10:10:34.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83f0511640 1 -- 172.21.15.174:0/1334092108 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e80b7b70 msgr2=0x7f83e8141630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83f0511640 1 --2- 172.21.15.174:0/1334092108 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e80b7b70 0x7f83e8141630 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f83e80b8370 tx=0x7f83e001c930 comp rx=0 tx=0).stop 2024-03-16T10:10:34.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83f0511640 1 -- 172.21.15.174:0/1334092108 shutdown_connections 2024-03-16T10:10:34.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83f0511640 1 --2- 172.21.15.174:0/1334092108 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e80b7b70 0x7f83e8141630 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83f0511640 1 --2- 172.21.15.174:0/1334092108 >> v2:172.21.15.174:3300/0 conn(0x7f83e8073070 0x7f83e81410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83f0511640 1 --2- 172.21.15.174:0/1334092108 >> v2:172.21.15.17:3300/0 conn(0x7f83e80b63a0 0x7f83e806d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83f0511640 1 -- 172.21.15.174:0/1334092108 >> 172.21.15.174:0/1334092108 conn(0x7f83e806b290 msgr2=0x7f83e806b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:34.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83f0511640 1 -- 172.21.15.174:0/1334092108 shutdown_connections 2024-03-16T10:10:34.822 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.818+0000 7f83f0511640 1 -- 172.21.15.174:0/1334092108 wait complete. 2024-03-16T10:10:34.825 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83f0511640 1 Processor -- start 2024-03-16T10:10:34.825 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83f0511640 1 -- start start 2024-03-16T10:10:34.825 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83f0511640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e8073070 0x7f83e80bae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.825 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83f0511640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e80b63a0 0x7f83e8143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.825 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83f0511640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b7b70 0x7f83e8143940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:34.825 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83f0511640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f83e80b5e80 con 0x7f83e80b7b70 2024-03-16T10:10:34.825 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83f0511640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83e80b5d00 con 0x7f83e80b63a0 2024-03-16T10:10:34.825 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83f0511640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f83e8147bf0 con 0x7f83e8073070 2024-03-16T10:10:34.826 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83ee2ad640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e8073070 0x7f83e80bae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.826 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83ee2ad640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e8073070 0x7f83e80bae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59212/0 (socket says 172.21.15.174:59212) 2024-03-16T10:10:34.826 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83ee2ad640 1 -- 172.21.15.174:0/643751319 learned_addr learned my addr 172.21.15.174:0/643751319 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-16T10:10:34.826 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83edaac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e80b63a0 0x7f83e8143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.826 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83eeaae640 1 --2- 172.21.15.174:0/643751319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b7b70 0x7f83e8143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:34.826 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83edaac640 1 -- 172.21.15.174:0/643751319 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e8073070 msgr2=0x7f83e80bae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.826 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83edaac640 1 --2- 172.21.15.174:0/643751319 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e8073070 0x7f83e80bae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.826 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83edaac640 1 -- 172.21.15.174:0/643751319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b7b70 msgr2=0x7f83e8143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.826 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83edaac640 1 --2- 172.21.15.174:0/643751319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b7b70 0x7f83e8143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.826 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83edaac640 1 -- 172.21.15.174:0/643751319 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e80aeab0 con 0x7f83e80b63a0 2024-03-16T10:10:34.826 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83ee2ad640 1 --2- 172.21.15.174:0/643751319 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e8073070 0x7f83e80bae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:10:34.826 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83edaac640 1 --2- 172.21.15.174:0/643751319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e80b63a0 0x7f83e8143400 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f83d800e910 tx=0x7f83d800edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:34.826 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83deffd640 1 -- 172.21.15.174:0/643751319 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d800ca00 con 0x7f83e80b63a0 2024-03-16T10:10:34.827 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83deffd640 1 -- 172.21.15.174:0/643751319 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83d800cba0 con 0x7f83e80b63a0 2024-03-16T10:10:34.827 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83deffd640 1 -- 172.21.15.174:0/643751319 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d8004030 con 0x7f83e80b63a0 2024-03-16T10:10:34.827 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83f0511640 1 -- 172.21.15.174:0/643751319 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e8142400 con 0x7f83e80b63a0 2024-03-16T10:10:34.827 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83f0511640 1 -- 172.21.15.174:0/643751319 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e80ae7d0 con 0x7f83e80b63a0 2024-03-16T10:10:34.827 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83deffd640 1 -- 172.21.15.174:0/643751319 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f83d80059a0 con 0x7f83e80b63a0 2024-03-16T10:10:34.827 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83deffd640 1 -- 172.21.15.174:0/643751319 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f83d80093d0 con 0x7f83e80b63a0 2024-03-16T10:10:34.827 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.822+0000 7f83f0511640 1 -- 172.21.15.174:0/643751319 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83e806a130 con 0x7f83e80b63a0 2024-03-16T10:10:34.827 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.826+0000 7f83deffd640 1 -- 172.21.15.174:0/643751319 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f83d8017020 con 0x7f83e80b63a0 2024-03-16T10:10:34.848 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:34.846+0000 7f2adf639180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T10:10:34.895 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.894+0000 7f83f0511640 1 -- 172.21.15.174:0/643751319 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ddb163f0-cce0-42a6-8d92-644babc32d58", "id": 4} v 0) -- 0x7f83e8068ad0 con 0x7f83e80b63a0 2024-03-16T10:10:34.900 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.898+0000 7f83deffd640 1 -- 172.21.15.174:0/643751319 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ddb163f0-cce0-42a6-8d92-644babc32d58", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f83d8003dd0 con 0x7f83e80b63a0 2024-03-16T10:10:34.900 INFO:teuthology.orchestra.run.smithi174.stdout:4 2024-03-16T10:10:34.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.898+0000 7f83f0511640 1 -- 172.21.15.174:0/643751319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e80b63a0 msgr2=0x7f83e8143400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:34.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.898+0000 7f83f0511640 1 --2- 172.21.15.174:0/643751319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e80b63a0 0x7f83e8143400 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f83d800e910 tx=0x7f83d800edd0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.898+0000 7f83f0511640 1 -- 172.21.15.174:0/643751319 shutdown_connections 2024-03-16T10:10:34.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.898+0000 7f83f0511640 1 --2- 172.21.15.174:0/643751319 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b7b70 0x7f83e8143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.898+0000 7f83f0511640 1 --2- 172.21.15.174:0/643751319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e80b63a0 0x7f83e8143400 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.898+0000 7f83f0511640 1 --2- 172.21.15.174:0/643751319 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e8073070 0x7f83e80bae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:34.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.898+0000 7f83f0511640 1 -- 172.21.15.174:0/643751319 >> 172.21.15.174:0/643751319 conn(0x7f83e806b290 msgr2=0x7f83e813a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:34.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.898+0000 7f83f0511640 1 -- 172.21.15.174:0/643751319 shutdown_connections 2024-03-16T10:10:34.902 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:34.898+0000 7f83f0511640 1 -- 172.21.15.174:0/643751319 wait complete. 2024-03-16T10:10:34.902 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:34.898+0000 7f2adf639180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T10:10:34.918 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 6e5d81da-6610-4194-91ca-b96d05e4ae48 5 2024-03-16T10:10:34.984 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:34.982+0000 7f2adf639180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T10:10:35.030 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.026+0000 7fcd9cf49640 1 Processor -- start 2024-03-16T10:10:35.030 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.026+0000 7fcd9cf49640 1 -- start start 2024-03-16T10:10:35.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9cf49640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcd980b1c60 0x7fcd98146da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9cf49640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcd980af8b0 0x7fcd98073030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9cf49640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd980b00b0 con 0x7fcd980b4b20 2024-03-16T10:10:35.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9cf49640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fcd9813dff0 con 0x7fcd980af8b0 2024-03-16T10:10:35.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9cf49640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fcd980b42b0 con 0x7fcd980b1c60 2024-03-16T10:10:35.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd96d9d640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fcd980b4b20 0x7fcd98143830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.174:55322/0 (socket says 172.21.15.174:55322) 2024-03-16T10:10:35.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9659c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcd980b1c60 0x7fcd98146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd96d9d640 1 -- 172.21.15.174:0/3236349134 learned_addr learned my addr 172.21.15.174:0/3236349134 (peer_addr_for_me v1:172.21.15.174:0/0) 2024-03-16T10:10:35.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd95d9b640 1 --2- 172.21.15.174:0/3236349134 >> v2:172.21.15.17:3300/0 conn(0x7fcd980af8b0 0x7fcd98073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9659c640 1 -- 172.21.15.174:0/3236349134 >> v2:172.21.15.17:3300/0 conn(0x7fcd980af8b0 msgr2=0x7fcd98073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:35.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9659c640 1 --2- 172.21.15.174:0/3236349134 >> v2:172.21.15.17:3300/0 conn(0x7fcd980af8b0 0x7fcd98073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.032 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9659c640 1 -- 172.21.15.174:0/3236349134 >> v1:172.21.15.174:6789/0 conn(0x7fcd980b4b20 legacy=0x7fcd98143830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9559a640 1 -- 172.21.15.174:0/3236349134 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 935144126 0 0) 0x7fcd980b00b0 con 0x7fcd980b4b20 2024-03-16T10:10:35.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9659c640 1 -- 172.21.15.174:0/3236349134 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd980b2040 con 0x7fcd980b1c60 2024-03-16T10:10:35.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9659c640 1 --2- 172.21.15.174:0/3236349134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980b1c60 0x7fcd98146da0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd980b9ce0 tx=0x7fcd8801c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3db461e3243c78db server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:35.033 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9559a640 1 -- 172.21.15.174:0/3236349134 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880b0070 con 0x7fcd980b1c60 2024-03-16T10:10:35.034 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9559a640 1 -- 172.21.15.174:0/3236349134 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd880ab600 con 0x7fcd980b1c60 2024-03-16T10:10:35.034 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9559a640 1 -- 172.21.15.174:0/3236349134 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880ab9d0 con 0x7fcd980b1c60 2024-03-16T10:10:35.034 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9cf49640 1 -- 172.21.15.174:0/3236349134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980b1c60 msgr2=0x7fcd98146da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.034 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9cf49640 1 --2- 172.21.15.174:0/3236349134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980b1c60 0x7fcd98146da0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd980b9ce0 tx=0x7fcd8801c3f0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9cf49640 1 -- 172.21.15.174:0/3236349134 shutdown_connections 2024-03-16T10:10:35.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9cf49640 1 --2- 172.21.15.174:0/3236349134 >> v2:172.21.15.17:3300/0 conn(0x7fcd980af8b0 0x7fcd98073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9cf49640 1 --2- 172.21.15.174:0/3236349134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980b1c60 0x7fcd98146da0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.030+0000 7fcd9cf49640 1 -- 172.21.15.174:0/3236349134 >> 172.21.15.174:0/3236349134 conn(0x7fcd980b8e80 msgr2=0x7fcd980b9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:35.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9cf49640 1 -- 172.21.15.174:0/3236349134 shutdown_connections 2024-03-16T10:10:35.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9cf49640 1 -- 172.21.15.174:0/3236349134 wait complete. 2024-03-16T10:10:35.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9cf49640 1 Processor -- start 2024-03-16T10:10:35.035 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9cf49640 1 -- start start 2024-03-16T10:10:35.038 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9cf49640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980af8b0 0x7fcd98146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.038 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9cf49640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd980b1c60 0x7fcd980adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.038 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9cf49640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd980b4b20 0x7fcd980ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.038 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9cf49640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcd980b57b0 con 0x7fcd980b4b20 2024-03-16T10:10:35.038 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9cf49640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd980b5630 con 0x7fcd980af8b0 2024-03-16T10:10:35.038 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9cf49640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fcd98073570 con 0x7fcd980b1c60 2024-03-16T10:10:35.038 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980af8b0 0x7fcd98146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.038 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd95d9b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd980b1c60 0x7fcd980adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.038 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980af8b0 0x7fcd98146520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40744/0 (socket says 172.21.15.174:40744) 2024-03-16T10:10:35.038 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9659c640 1 -- 172.21.15.174:0/1712847847 learned_addr learned my addr 172.21.15.174:0/1712847847 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd95d9b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd980b1c60 0x7fcd980adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59216/0 (socket says 172.21.15.174:59216) 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd96d9d640 1 --2- 172.21.15.174:0/1712847847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd980b4b20 0x7fcd980ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9659c640 1 -- 172.21.15.174:0/1712847847 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd980b1c60 msgr2=0x7fcd980adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9659c640 1 --2- 172.21.15.174:0/1712847847 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd980b1c60 0x7fcd980adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9659c640 1 -- 172.21.15.174:0/1712847847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd980b4b20 msgr2=0x7fcd980ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9659c640 1 --2- 172.21.15.174:0/1712847847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd980b4b20 0x7fcd980ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9659c640 1 -- 172.21.15.174:0/1712847847 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd98070b20 con 0x7fcd980af8b0 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9659c640 1 --2- 172.21.15.174:0/1712847847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980af8b0 0x7fcd98146520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd88019000 tx=0x7fcd88019190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd8effd640 1 -- 172.21.15.174:0/1712847847 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880b0040 con 0x7fcd980af8b0 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd8effd640 1 -- 172.21.15.174:0/1712847847 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd880b9070 con 0x7fcd980af8b0 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9cf49640 1 -- 172.21.15.174:0/1712847847 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd980af1d0 con 0x7fcd980af8b0 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd8effd640 1 -- 172.21.15.174:0/1712847847 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd880b4610 con 0x7fcd980af8b0 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9cf49640 1 -- 172.21.15.174:0/1712847847 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd980705d0 con 0x7fcd980af8b0 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd8effd640 1 -- 172.21.15.174:0/1712847847 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcd880b47b0 con 0x7fcd980af8b0 2024-03-16T10:10:35.039 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd8effd640 1 -- 172.21.15.174:0/1712847847 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fcd880bd3c0 con 0x7fcd980af8b0 2024-03-16T10:10:35.040 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.034+0000 7fcd9cf49640 1 -- 172.21.15.174:0/1712847847 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd9806a130 con 0x7fcd980af8b0 2024-03-16T10:10:35.040 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:35.034+0000 7f2adf639180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T10:10:35.040 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.038+0000 7fcd8effd640 1 -- 172.21.15.174:0/1712847847 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcd88016070 con 0x7fcd980af8b0 2024-03-16T10:10:35.093 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:35.090+0000 7faa7f5b9180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T10:10:35.108 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.106+0000 7fcd9cf49640 1 -- 172.21.15.174:0/1712847847 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6e5d81da-6610-4194-91ca-b96d05e4ae48", "id": 5} v 0) -- 0x7fcd980b8220 con 0x7fcd980af8b0 2024-03-16T10:10:35.113 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.110+0000 7fcd8effd640 1 -- 172.21.15.174:0/1712847847 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6e5d81da-6610-4194-91ca-b96d05e4ae48", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fcd880be460 con 0x7fcd980af8b0 2024-03-16T10:10:35.113 INFO:teuthology.orchestra.run.smithi174.stdout:5 2024-03-16T10:10:35.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.114+0000 7fcd9cf49640 1 -- 172.21.15.174:0/1712847847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980af8b0 msgr2=0x7fcd98146520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.114+0000 7fcd9cf49640 1 --2- 172.21.15.174:0/1712847847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980af8b0 0x7fcd98146520 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcd88019000 tx=0x7fcd88019190 comp rx=0 tx=0).stop 2024-03-16T10:10:35.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.114+0000 7fcd9cf49640 1 -- 172.21.15.174:0/1712847847 shutdown_connections 2024-03-16T10:10:35.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.114+0000 7fcd9cf49640 1 --2- 172.21.15.174:0/1712847847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd980b4b20 0x7fcd980ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.114+0000 7fcd9cf49640 1 --2- 172.21.15.174:0/1712847847 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd980b1c60 0x7fcd980adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.114+0000 7fcd9cf49640 1 --2- 172.21.15.174:0/1712847847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd980af8b0 0x7fcd98146520 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.114+0000 7fcd9cf49640 1 -- 172.21.15.174:0/1712847847 >> 172.21.15.174:0/1712847847 conn(0x7fcd980b8e80 msgr2=0x7fcd98143320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:35.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.114+0000 7fcd9cf49640 1 -- 172.21.15.174:0/1712847847 shutdown_connections 2024-03-16T10:10:35.115 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.114+0000 7fcd9cf49640 1 -- 172.21.15.174:0/1712847847 wait complete. 2024-03-16T10:10:35.116 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:35.114+0000 7f2adf639180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T10:10:35.130 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 0f5f0885-f9c9-4ddc-9cf0-216197b35186 6 2024-03-16T10:10:35.148 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:35.146+0000 7faa7f5b9180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T10:10:35.172 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:35.170+0000 7f2adf639180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T10:10:35.203 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:35.202+0000 7faa7f5b9180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T10:10:35.245 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.242+0000 7fab51e33640 1 Processor -- start 2024-03-16T10:10:35.246 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.242+0000 7fab51e33640 1 -- start start 2024-03-16T10:10:35.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab51e33640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab4c0b5740 0x7fab4c0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab51e33640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fab4c14c9a0 0x7fab4c14ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab51e33640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab4c0b5b20 con 0x7fab4c06ede0 2024-03-16T10:10:35.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab51e33640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fab4c06e950 con 0x7fab4c14c9a0 2024-03-16T10:10:35.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab51e33640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fab4c0af720 con 0x7fab4c0b5740 2024-03-16T10:10:35.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab4b7fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab4c0b5740 0x7fab4c0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab4bfff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fab4c14c9a0 0x7fab4c14ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab4b7fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab4c0b5740 0x7fab4c0aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59232/0 (socket says 172.21.15.174:59232) 2024-03-16T10:10:35.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab4b7fe640 1 -- 172.21.15.174:0/4003593679 learned_addr learned my addr 172.21.15.174:0/4003593679 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-16T10:10:35.247 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab4bfff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fab4c14c9a0 0x7fab4c14ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40750/0 (socket says 172.21.15.174:40750) 2024-03-16T10:10:35.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab4b7fe640 1 -- 172.21.15.174:0/4003593679 >> v2:172.21.15.174:3300/0 conn(0x7fab4c14c9a0 msgr2=0x7fab4c14ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:35.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab4b7fe640 1 --2- 172.21.15.174:0/4003593679 >> v2:172.21.15.174:3300/0 conn(0x7fab4c14c9a0 0x7fab4c14ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab4b7fe640 1 -- 172.21.15.174:0/4003593679 >> v1:172.21.15.17:6789/0 conn(0x7fab4c06ede0 legacy=0x7fab4c071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab4b7fe640 1 -- 172.21.15.174:0/4003593679 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4c0afa00 con 0x7fab4c0b5740 2024-03-16T10:10:35.248 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab4bfff640 1 --2- 172.21.15.174:0/4003593679 >> v2:172.21.15.174:3300/0 conn(0x7fab4c14c9a0 0x7fab4c14ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:10:35.249 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab4b7fe640 1 --2- 172.21.15.174:0/4003593679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab4c0b5740 0x7fab4c0aeaf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fab4000bb40 tx=0x7fab4001ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d28912e90051217 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:35.249 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab4a7fc640 1 -- 172.21.15.174:0/4003593679 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab400b4070 con 0x7fab4c0b5740 2024-03-16T10:10:35.249 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab4a7fc640 1 -- 172.21.15.174:0/4003593679 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab400a7af0 con 0x7fab4c0b5740 2024-03-16T10:10:35.249 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab4a7fc640 1 -- 172.21.15.174:0/4003593679 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab400af440 con 0x7fab4c0b5740 2024-03-16T10:10:35.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab51e33640 1 -- 172.21.15.174:0/4003593679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab4c0b5740 msgr2=0x7fab4c0aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab51e33640 1 --2- 172.21.15.174:0/4003593679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab4c0b5740 0x7fab4c0aeaf0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fab4000bb40 tx=0x7fab4001ec10 comp rx=0 tx=0).stop 2024-03-16T10:10:35.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab51e33640 1 -- 172.21.15.174:0/4003593679 shutdown_connections 2024-03-16T10:10:35.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab51e33640 1 --2- 172.21.15.174:0/4003593679 >> v2:172.21.15.174:3300/0 conn(0x7fab4c14c9a0 0x7fab4c14ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab51e33640 1 --2- 172.21.15.174:0/4003593679 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab4c0b5740 0x7fab4c0aeaf0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.250 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab51e33640 1 -- 172.21.15.174:0/4003593679 >> 172.21.15.174:0/4003593679 conn(0x7fab4c0b65d0 msgr2=0x7fab4c0b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:35.251 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab51e33640 1 -- 172.21.15.174:0/4003593679 shutdown_connections 2024-03-16T10:10:35.251 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.246+0000 7fab51e33640 1 -- 172.21.15.174:0/4003593679 wait complete. 2024-03-16T10:10:35.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab51e33640 1 Processor -- start 2024-03-16T10:10:35.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab51e33640 1 -- start start 2024-03-16T10:10:35.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab51e33640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab4c06ede0 0x7fab4c1dfa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab51e33640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab4c0b5740 0x7fab4c09f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab51e33640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab4c14c9a0 0x7fab4c1e3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab51e33640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c0725e0 con 0x7fab4c14c9a0 2024-03-16T10:10:35.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab51e33640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fab4c072760 con 0x7fab4c06ede0 2024-03-16T10:10:35.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab51e33640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fab4c06f1c0 con 0x7fab4c0b5740 2024-03-16T10:10:35.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab4b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab4c06ede0 0x7fab4c1dfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab4affd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab4c0b5740 0x7fab4c09f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.253 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab4b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab4c06ede0 0x7fab4c1dfa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40762/0 (socket says 172.21.15.174:40762) 2024-03-16T10:10:35.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab4affd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab4c0b5740 0x7fab4c09f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59242/0 (socket says 172.21.15.174:59242) 2024-03-16T10:10:35.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab4b7fe640 1 -- 172.21.15.174:0/2341067377 learned_addr learned my addr 172.21.15.174:0/2341067377 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-16T10:10:35.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab4bfff640 1 --2- 172.21.15.174:0/2341067377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab4c14c9a0 0x7fab4c1e3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab4affd640 1 -- 172.21.15.174:0/2341067377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab4c06ede0 msgr2=0x7fab4c1dfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab4affd640 1 --2- 172.21.15.174:0/2341067377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab4c06ede0 0x7fab4c1dfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab4affd640 1 -- 172.21.15.174:0/2341067377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab4c14c9a0 msgr2=0x7fab4c1e3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab4affd640 1 --2- 172.21.15.174:0/2341067377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab4c14c9a0 0x7fab4c1e3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab4affd640 1 -- 172.21.15.174:0/2341067377 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4c14efe0 con 0x7fab4c0b5740 2024-03-16T10:10:35.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab4b7fe640 1 --2- 172.21.15.174:0/2341067377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab4c06ede0 0x7fab4c1dfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:10:35.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab4affd640 1 --2- 172.21.15.174:0/2341067377 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab4c0b5740 0x7fab4c09f0b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fab3c000f70 tx=0x7fab3c00c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:35.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab33fff640 1 -- 172.21.15.174:0/2341067377 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c017070 con 0x7fab4c0b5740 2024-03-16T10:10:35.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab33fff640 1 -- 172.21.15.174:0/2341067377 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab3c0052d0 con 0x7fab4c0b5740 2024-03-16T10:10:35.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab51e33640 1 -- 172.21.15.174:0/2341067377 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4c1e0070 con 0x7fab4c0b5740 2024-03-16T10:10:35.254 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab33fff640 1 -- 172.21.15.174:0/2341067377 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3c0054c0 con 0x7fab4c0b5740 2024-03-16T10:10:35.255 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab51e33640 1 -- 172.21.15.174:0/2341067377 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4c1e0630 con 0x7fab4c0b5740 2024-03-16T10:10:35.255 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab33fff640 1 -- 172.21.15.174:0/2341067377 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fab3c01a070 con 0x7fab4c0b5740 2024-03-16T10:10:35.255 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab33fff640 1 -- 172.21.15.174:0/2341067377 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fab3c011b80 con 0x7fab4c0b5740 2024-03-16T10:10:35.255 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.250+0000 7fab51e33640 1 -- 172.21.15.174:0/2341067377 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab4c06a130 con 0x7fab4c0b5740 2024-03-16T10:10:35.255 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.254+0000 7fab33fff640 1 -- 172.21.15.174:0/2341067377 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fab3c021020 con 0x7fab4c0b5740 2024-03-16T10:10:35.284 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:35.282+0000 7faa7f5b9180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T10:10:35.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.322+0000 7fab51e33640 1 -- 172.21.15.174:0/2341067377 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0f5f0885-f9c9-4ddc-9cf0-216197b35186", "id": 6} v 0) -- 0x7fab4c1e0800 con 0x7fab4c0b5740 2024-03-16T10:10:35.327 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:35.326+0000 7f2adf639180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T10:10:35.331 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.330+0000 7fab33fff640 1 -- 172.21.15.174:0/2341067377 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0f5f0885-f9c9-4ddc-9cf0-216197b35186", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fab3c011590 con 0x7fab4c0b5740 2024-03-16T10:10:35.331 INFO:teuthology.orchestra.run.smithi174.stdout:6 2024-03-16T10:10:35.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.330+0000 7fab51e33640 1 -- 172.21.15.174:0/2341067377 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab4c0b5740 msgr2=0x7fab4c09f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.330+0000 7fab51e33640 1 --2- 172.21.15.174:0/2341067377 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab4c0b5740 0x7fab4c09f0b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fab3c000f70 tx=0x7fab3c00c890 comp rx=0 tx=0).stop 2024-03-16T10:10:35.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.330+0000 7fab51e33640 1 -- 172.21.15.174:0/2341067377 shutdown_connections 2024-03-16T10:10:35.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.330+0000 7fab51e33640 1 --2- 172.21.15.174:0/2341067377 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab4c14c9a0 0x7fab4c1e3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.330+0000 7fab51e33640 1 --2- 172.21.15.174:0/2341067377 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab4c0b5740 0x7fab4c09f0b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.330+0000 7fab51e33640 1 --2- 172.21.15.174:0/2341067377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab4c06ede0 0x7fab4c1dfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.330+0000 7fab51e33640 1 -- 172.21.15.174:0/2341067377 >> 172.21.15.174:0/2341067377 conn(0x7fab4c0b65d0 msgr2=0x7fab4c13e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:35.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.330+0000 7fab51e33640 1 -- 172.21.15.174:0/2341067377 shutdown_connections 2024-03-16T10:10:35.333 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.330+0000 7fab51e33640 1 -- 172.21.15.174:0/2341067377 wait complete. 2024-03-16T10:10:35.338 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:35.338+0000 7faa7f5b9180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T10:10:35.348 DEBUG:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new c133836d-5f03-4ecd-9876-14aa5f99b990 7 2024-03-16T10:10:35.391 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:35.390+0000 7f2adf639180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T10:10:35.419 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:35.418+0000 7faa7f5b9180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T10:10:35.474 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:35.474+0000 7faa7f5b9180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T10:10:35.511 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41bdc2640 1 Processor -- start 2024-03-16T10:10:35.511 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41bdc2640 1 -- start start 2024-03-16T10:10:35.512 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41bdc2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc414067220 0x7fc4141478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.512 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41bdc2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4140b4e50 0x7fc414147e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41bdc2640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc4141414d0 0x7fc4141418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41bdc2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc4140af150 con 0x7fc414067220 2024-03-16T10:10:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41bdc2640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fc4140aeee0 con 0x7fc4141414d0 2024-03-16T10:10:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41bdc2640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc414067600 con 0x7fc4140b4e50 2024-03-16T10:10:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41a35f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc4141414d0 0x7fc4141418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41935d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4140b4e50 0x7fc414147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41a35f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc4141414d0 0x7fc4141418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40778/0 (socket says 172.21.15.174:40778) 2024-03-16T10:10:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41a35f640 1 -- 172.21.15.174:0/2561822057 learned_addr learned my addr 172.21.15.174:0/2561822057 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-16T10:10:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41935d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4140b4e50 0x7fc414147e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59248/0 (socket says 172.21.15.174:59248) 2024-03-16T10:10:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc419b5e640 1 --2- 172.21.15.174:0/2561822057 >> v2:172.21.15.17:3300/0 conn(0x7fc414067220 0x7fc4141478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41a35f640 1 -- 172.21.15.174:0/2561822057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4140b4e50 msgr2=0x7fc414147e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41a35f640 1 --2- 172.21.15.174:0/2561822057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4140b4e50 0x7fc414147e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41a35f640 1 -- 172.21.15.174:0/2561822057 >> v2:172.21.15.17:3300/0 conn(0x7fc414067220 msgr2=0x7fc4141478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41a35f640 1 --2- 172.21.15.174:0/2561822057 >> v2:172.21.15.17:3300/0 conn(0x7fc414067220 0x7fc4141478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41a35f640 1 -- 172.21.15.174:0/2561822057 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4140b5400 con 0x7fc4141414d0 2024-03-16T10:10:35.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.510+0000 7fc41935d640 1 --2- 172.21.15.174:0/2561822057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4140b4e50 0x7fc414147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:10:35.514 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41a35f640 1 --2- 172.21.15.174:0/2561822057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4141414d0 0x7fc4141418e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc4140ae460 tx=0x7fc41001eb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7b2a568045d8a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:35.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc418b5c640 1 -- 172.21.15.174:0/2561822057 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4100b4070 con 0x7fc4141414d0 2024-03-16T10:10:35.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc418b5c640 1 -- 172.21.15.174:0/2561822057 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4100a7b00 con 0x7fc4141414d0 2024-03-16T10:10:35.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc418b5c640 1 -- 172.21.15.174:0/2561822057 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4100af440 con 0x7fc4141414d0 2024-03-16T10:10:35.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 -- 172.21.15.174:0/2561822057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4141414d0 msgr2=0x7fc4141418e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 --2- 172.21.15.174:0/2561822057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4141414d0 0x7fc4141418e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc4140ae460 tx=0x7fc41001eb60 comp rx=0 tx=0).stop 2024-03-16T10:10:35.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 -- 172.21.15.174:0/2561822057 shutdown_connections 2024-03-16T10:10:35.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 --2- 172.21.15.174:0/2561822057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4141414d0 0x7fc4141418e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 --2- 172.21.15.174:0/2561822057 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4140b4e50 0x7fc414147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 --2- 172.21.15.174:0/2561822057 >> v2:172.21.15.17:3300/0 conn(0x7fc414067220 0x7fc4141478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 -- 172.21.15.174:0/2561822057 >> 172.21.15.174:0/2561822057 conn(0x7fc41406bd70 msgr2=0x7fc4140b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:35.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 -- 172.21.15.174:0/2561822057 shutdown_connections 2024-03-16T10:10:35.516 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 -- 172.21.15.174:0/2561822057 wait complete. 2024-03-16T10:10:35.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 Processor -- start 2024-03-16T10:10:35.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 -- start start 2024-03-16T10:10:35.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc414067220 0x7fc414139310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4140b4e50 0x7fc4141df980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4141414d0 0x7fc4140b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc41409e890 con 0x7fc4141414d0 2024-03-16T10:10:35.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc41409e710 con 0x7fc414067220 2024-03-16T10:10:35.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc4140afdf0 con 0x7fc4140b4e50 2024-03-16T10:10:35.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41935d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4140b4e50 0x7fc4141df980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc419b5e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc414067220 0x7fc414139310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41935d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4140b4e50 0x7fc4141df980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.174:59264/0 (socket says 172.21.15.174:59264) 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc419b5e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc414067220 0x7fc414139310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40794/0 (socket says 172.21.15.174:40794) 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41935d640 1 -- 172.21.15.174:0/2737219452 learned_addr learned my addr 172.21.15.174:0/2737219452 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41a35f640 1 --2- 172.21.15.174:0/2737219452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4141414d0 0x7fc4140b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc419b5e640 1 -- 172.21.15.174:0/2737219452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4140b4e50 msgr2=0x7fc4141df980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc419b5e640 1 --2- 172.21.15.174:0/2737219452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4140b4e50 0x7fc4141df980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc419b5e640 1 -- 172.21.15.174:0/2737219452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4141414d0 msgr2=0x7fc4140b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc419b5e640 1 --2- 172.21.15.174:0/2737219452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4141414d0 0x7fc4140b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc419b5e640 1 -- 172.21.15.174:0/2737219452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4140b3740 con 0x7fc414067220 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41a35f640 1 --2- 172.21.15.174:0/2737219452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4141414d0 0x7fc4140b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc419b5e640 1 --2- 172.21.15.174:0/2737219452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc414067220 0x7fc414139310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc40c007e60 tx=0x7fc40c00e4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc40a7fc640 1 -- 172.21.15.174:0/2737219452 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c014620 con 0x7fc414067220 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc40a7fc640 1 -- 172.21.15.174:0/2737219452 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40c019070 con 0x7fc414067220 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 -- 172.21.15.174:0/2737219452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4140b8fb0 con 0x7fc414067220 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc40a7fc640 1 -- 172.21.15.174:0/2737219452 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c0149e0 con 0x7fc414067220 2024-03-16T10:10:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc41bdc2640 1 -- 172.21.15.174:0/2737219452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4141dfec0 con 0x7fc414067220 2024-03-16T10:10:35.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc40a7fc640 1 -- 172.21.15.174:0/2737219452 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc40c014b80 con 0x7fc414067220 2024-03-16T10:10:35.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.514+0000 7fc40a7fc640 1 -- 172.21.15.174:0/2737219452 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fc40c01da20 con 0x7fc414067220 2024-03-16T10:10:35.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.518+0000 7fc41bdc2640 1 -- 172.21.15.174:0/2737219452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc41406a130 con 0x7fc414067220 2024-03-16T10:10:35.521 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.518+0000 7fc40a7fc640 1 -- 172.21.15.174:0/2737219452 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc40c01dc20 con 0x7fc414067220 2024-03-16T10:10:35.588 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.586+0000 7fc41bdc2640 1 -- 172.21.15.174:0/2737219452 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c133836d-5f03-4ecd-9876-14aa5f99b990", "id": 7} v 0) -- 0x7fc4140aeb90 con 0x7fc414067220 2024-03-16T10:10:35.597 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.594+0000 7fc40a7fc640 1 -- 172.21.15.174:0/2737219452 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c133836d-5f03-4ecd-9876-14aa5f99b990", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fc40c01de00 con 0x7fc414067220 2024-03-16T10:10:35.597 INFO:teuthology.orchestra.run.smithi174.stdout:7 2024-03-16T10:10:35.598 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.594+0000 7fc41bdc2640 1 -- 172.21.15.174:0/2737219452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc414067220 msgr2=0x7fc414139310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.598 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.594+0000 7fc41bdc2640 1 --2- 172.21.15.174:0/2737219452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc414067220 0x7fc414139310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc40c007e60 tx=0x7fc40c00e4b0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.598 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.594+0000 7fc41bdc2640 1 -- 172.21.15.174:0/2737219452 shutdown_connections 2024-03-16T10:10:35.598 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.594+0000 7fc41bdc2640 1 --2- 172.21.15.174:0/2737219452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4141414d0 0x7fc4140b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.598 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.594+0000 7fc41bdc2640 1 --2- 172.21.15.174:0/2737219452 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4140b4e50 0x7fc4141df980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.598 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.594+0000 7fc41bdc2640 1 --2- 172.21.15.174:0/2737219452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc414067220 0x7fc414139310 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.598 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.594+0000 7fc41bdc2640 1 -- 172.21.15.174:0/2737219452 >> 172.21.15.174:0/2737219452 conn(0x7fc41406bd70 msgr2=0x7fc41409aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:35.599 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.594+0000 7fc41bdc2640 1 -- 172.21.15.174:0/2737219452 shutdown_connections 2024-03-16T10:10:35.599 INFO:teuthology.orchestra.run.smithi174.stderr:2024-03-16T10:10:35.594+0000 7fc41bdc2640 1 -- 172.21.15.174:0/2737219452 wait complete. 2024-03-16T10:10:35.612 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-16T10:10:35.613 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-16T10:10:35.615 INFO:tasks.ceph.osd.0:Started 2024-03-16T10:10:35.615 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-16T10:10:35.615 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-16T10:10:35.617 INFO:tasks.ceph.osd.1:Started 2024-03-16T10:10:35.618 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-16T10:10:35.618 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-16T10:10:35.620 INFO:tasks.ceph.osd.2:Started 2024-03-16T10:10:35.620 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-16T10:10:35.620 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-16T10:10:35.622 INFO:tasks.ceph.osd.3:Started 2024-03-16T10:10:35.622 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-16T10:10:35.622 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-16T10:10:35.624 INFO:tasks.ceph.osd.4:Started 2024-03-16T10:10:35.624 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-16T10:10:35.625 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-16T10:10:35.626 INFO:tasks.ceph.osd.5:Started 2024-03-16T10:10:35.626 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-16T10:10:35.626 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-16T10:10:35.628 INFO:tasks.ceph.osd.6:Started 2024-03-16T10:10:35.628 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-16T10:10:35.628 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-16T10:10:35.630 INFO:tasks.ceph.osd.7:Started 2024-03-16T10:10:35.630 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:10:35.631 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:35.630+0000 7faa7f5b9180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T10:10:35.717 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:35.718+0000 7faa7f5b9180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T10:10:35.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb98e11640 1 Processor -- start 2024-03-16T10:10:35.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb98e11640 1 -- start start 2024-03-16T10:10:35.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb98e11640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb940b4650 0x7fdb9414b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb98e11640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdb940b7650 0x7fdb941412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb98e11640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb94141c50 con 0x7fdb94141820 2024-03-16T10:10:35.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb98e11640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fdb940b4a30 con 0x7fdb940b7650 2024-03-16T10:10:35.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb98e11640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9414abe0 con 0x7fdb940b4650 2024-03-16T10:10:35.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb91d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdb940b7650 0x7fdb941412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb92d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fdb94141820 0x7fdb941474d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52858/0 (socket says 172.21.15.17:52858) 2024-03-16T10:10:35.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb92d9d640 1 -- 172.21.15.17:0/23398703 learned_addr learned my addr 172.21.15.17:0/23398703 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:10:35.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb91d9b640 1 --2- 172.21.15.17:0/23398703 >> v2:172.21.15.17:3300/0 conn(0x7fdb940b7650 0x7fdb941412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54258/0 (socket says 172.21.15.17:54258) 2024-03-16T10:10:35.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb9259c640 1 --2- 172.21.15.17:0/23398703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb940b4650 0x7fdb9414b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb91d9b640 1 -- 172.21.15.17:0/23398703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb940b4650 msgr2=0x7fdb9414b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:35.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb91d9b640 1 --2- 172.21.15.17:0/23398703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb940b4650 0x7fdb9414b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb91d9b640 1 -- 172.21.15.17:0/23398703 >> v1:172.21.15.17:6789/0 conn(0x7fdb94141820 legacy=0x7fdb941474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb91d9b640 1 -- 172.21.15.17:0/23398703 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9414aec0 con 0x7fdb940b7650 2024-03-16T10:10:35.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb91d9b640 1 --2- 172.21.15.17:0/23398703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb940b7650 0x7fdb941412e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdb880097c0 tx=0x7fdb8801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a76149f52ba543 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:35.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb9159a640 1 -- 172.21.15.17:0/23398703 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb880b0070 con 0x7fdb940b7650 2024-03-16T10:10:35.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb9159a640 1 -- 172.21.15.17:0/23398703 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb880ab600 con 0x7fdb940b7650 2024-03-16T10:10:35.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.758+0000 7fdb9159a640 1 -- 172.21.15.17:0/23398703 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb880ab9d0 con 0x7fdb940b7650 2024-03-16T10:10:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 -- 172.21.15.17:0/23398703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb940b7650 msgr2=0x7fdb941412e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 --2- 172.21.15.17:0/23398703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb940b7650 0x7fdb941412e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdb880097c0 tx=0x7fdb8801c410 comp rx=0 tx=0).stop 2024-03-16T10:10:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 -- 172.21.15.17:0/23398703 shutdown_connections 2024-03-16T10:10:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 --2- 172.21.15.17:0/23398703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb940b7650 0x7fdb941412e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 --2- 172.21.15.17:0/23398703 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb940b4650 0x7fdb9414b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 -- 172.21.15.17:0/23398703 >> 172.21.15.17:0/23398703 conn(0x7fdb940bccd0 msgr2=0x7fdb940bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 -- 172.21.15.17:0/23398703 shutdown_connections 2024-03-16T10:10:35.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 -- 172.21.15.17:0/23398703 wait complete. 2024-03-16T10:10:35.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 Processor -- start 2024-03-16T10:10:35.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 -- start start 2024-03-16T10:10:35.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb940b4650 0x7fdb9413e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb940b7650 0x7fdb94142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb94141820 0x7fdb94142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:35.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdb940b94a0 con 0x7fdb94141820 2024-03-16T10:10:35.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb940b9320 con 0x7fdb940b7650 2024-03-16T10:10:35.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fdb940b59f0 con 0x7fdb940b4650 2024-03-16T10:10:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb92d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb94141820 0x7fdb94142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb92d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb94141820 0x7fdb94142ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54270/0 (socket says 172.21.15.17:54270) 2024-03-16T10:10:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb92d9d640 1 -- 172.21.15.17:0/4264105730 learned_addr learned my addr 172.21.15.17:0/4264105730 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb9259c640 1 --2- 172.21.15.17:0/4264105730 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb940b4650 0x7fdb9413e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb91d9b640 1 --2- 172.21.15.17:0/4264105730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb940b7650 0x7fdb94142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb92d9d640 1 -- 172.21.15.17:0/4264105730 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb940b4650 msgr2=0x7fdb9413e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb92d9d640 1 --2- 172.21.15.17:0/4264105730 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb940b4650 0x7fdb9413e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb92d9d640 1 -- 172.21.15.17:0/4264105730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb940b7650 msgr2=0x7fdb94142660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb92d9d640 1 --2- 172.21.15.17:0/4264105730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb940b7650 0x7fdb94142660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb92d9d640 1 -- 172.21.15.17:0/4264105730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb94144300 con 0x7fdb94141820 2024-03-16T10:10:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb92d9d640 1 --2- 172.21.15.17:0/4264105730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb94141820 0x7fdb94142ba0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00ca80 tx=0x7fdb8c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb82ffd640 1 -- 172.21.15.17:0/4264105730 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c017070 con 0x7fdb94141820 2024-03-16T10:10:35.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb82ffd640 1 -- 172.21.15.17:0/4264105730 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb8c00ac30 con 0x7fdb94141820 2024-03-16T10:10:35.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 -- 172.21.15.17:0/4264105730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb94146d60 con 0x7fdb94141820 2024-03-16T10:10:35.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb82ffd640 1 -- 172.21.15.17:0/4264105730 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c012600 con 0x7fdb94141820 2024-03-16T10:10:35.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.762+0000 7fdb98e11640 1 -- 172.21.15.17:0/4264105730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb94147320 con 0x7fdb94141820 2024-03-16T10:10:35.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.766+0000 7fdb82ffd640 1 -- 172.21.15.17:0/4264105730 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdb8c01a070 con 0x7fdb94141820 2024-03-16T10:10:35.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.766+0000 7fdb82ffd640 1 -- 172.21.15.17:0/4264105730 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fdb8c007070 con 0x7fdb94141820 2024-03-16T10:10:35.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.766+0000 7fdb98e11640 1 -- 172.21.15.17:0/4264105730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb94068360 con 0x7fdb94141820 2024-03-16T10:10:35.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.778+0000 7fdb82ffd640 1 -- 172.21.15.17:0/4264105730 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdb8c0127a0 con 0x7fdb94141820 2024-03-16T10:10:35.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.834+0000 7fdb98e11640 1 -- 172.21.15.17:0/4264105730 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb940b90c0 con 0x7fdb94141820 2024-03-16T10:10:35.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.834+0000 7fdb82ffd640 1 -- 172.21.15.17:0/4264105730 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fdb8c01c7b0 con 0x7fdb94141820 2024-03-16T10:10:35.836 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:35.836 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":10,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:35.591690+0000","last_up_change":"0.000000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:10:35.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.834+0000 7fdb98e11640 1 -- 172.21.15.17:0/4264105730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb94141820 msgr2=0x7fdb94142ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:35.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.834+0000 7fdb98e11640 1 --2- 172.21.15.17:0/4264105730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb94141820 0x7fdb94142ba0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00ca80 tx=0x7fdb8c00cf40 comp rx=0 tx=0).stop 2024-03-16T10:10:35.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.834+0000 7fdb98e11640 1 -- 172.21.15.17:0/4264105730 shutdown_connections 2024-03-16T10:10:35.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.834+0000 7fdb98e11640 1 --2- 172.21.15.17:0/4264105730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb94141820 0x7fdb94142ba0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.834+0000 7fdb98e11640 1 --2- 172.21.15.17:0/4264105730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb940b7650 0x7fdb94142660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.834+0000 7fdb98e11640 1 --2- 172.21.15.17:0/4264105730 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdb940b4650 0x7fdb9413e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.834+0000 7fdb98e11640 1 -- 172.21.15.17:0/4264105730 >> 172.21.15.17:0/4264105730 conn(0x7fdb940bccd0 msgr2=0x7fdb9413ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.834+0000 7fdb98e11640 1 -- 172.21.15.17:0/4264105730 shutdown_connections 2024-03-16T10:10:35.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:35.834+0000 7fdb98e11640 1 -- 172.21.15.17:0/4264105730 wait complete. 2024-03-16T10:10:35.852 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-16T10:10:35.852 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-16T10:10:36.063 INFO:tasks.ceph.mgr.x.smithi174.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-16T10:10:36.063 INFO:tasks.ceph.mgr.x.smithi174.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-16T10:10:36.063 INFO:tasks.ceph.mgr.x.smithi174.stderr: from numpy import show_config as show_numpy_config 2024-03-16T10:10:36.110 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:36.106+0000 7f2adf639180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T10:10:36.244 INFO:tasks.ceph.osd.5.smithi174.stderr:2024-03-16T10:10:36.242+0000 7f063160e7c0 -1 Falling back to public interface 2024-03-16T10:10:36.244 INFO:tasks.ceph.osd.6.smithi174.stderr:2024-03-16T10:10:36.242+0000 7fb6010537c0 -1 Falling back to public interface 2024-03-16T10:10:36.256 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-03-16T10:10:36.254+0000 7f5f8579c7c0 -1 Falling back to public interface 2024-03-16T10:10:36.256 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-03-16T10:10:36.254+0000 7fa44e3147c0 -1 Falling back to public interface 2024-03-16T10:10:36.256 INFO:tasks.ceph.osd.3.smithi017.stderr:2024-03-16T10:10:36.254+0000 7f0649d387c0 -1 Falling back to public interface 2024-03-16T10:10:36.256 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-03-16T10:10:36.254+0000 7fa44d2d4640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-03-16T10:10:36.259 INFO:tasks.ceph.osd.7.smithi174.stderr:2024-03-16T10:10:36.258+0000 7f6f45f3e7c0 -1 Falling back to public interface 2024-03-16T10:10:36.259 INFO:tasks.ceph.osd.4.smithi174.stderr:2024-03-16T10:10:36.258+0000 7fc3729627c0 -1 Falling back to public interface 2024-03-16T10:10:36.260 INFO:tasks.ceph.osd.4.smithi174.stderr:2024-03-16T10:10:36.258+0000 7fc371922640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6818/0: (98) Address already in use 2024-03-16T10:10:36.271 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-03-16T10:10:36.270+0000 7f9c1c9a67c0 -1 Falling back to public interface 2024-03-16T10:10:36.274 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:36.270+0000 7f2adf639180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T10:10:36.343 INFO:tasks.ceph.mgr.y.smithi017.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-16T10:10:36.343 INFO:tasks.ceph.mgr.y.smithi017.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-16T10:10:36.343 INFO:tasks.ceph.mgr.y.smithi017.stderr: from numpy import show_config as show_numpy_config 2024-03-16T10:10:36.385 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:36.382+0000 7faa7f5b9180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T10:10:36.452 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:36.450+0000 7f2adf639180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T10:10:36.522 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:36.518+0000 7f2adf639180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T10:10:36.534 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:36.534+0000 7faa7f5b9180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T10:10:36.658 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-16T10:10:36.669 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:36.666+0000 7f2adf639180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T10:10:36.699 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:36.698+0000 7faa7f5b9180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T10:10:36.767 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:36.766+0000 7faa7f5b9180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T10:10:36.773 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd810c640 1 Processor -- start 2024-03-16T10:10:36.774 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd810c640 1 -- start start 2024-03-16T10:10:36.777 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd810c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fbdd00b6ba0 0x7fbdd00721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:36.777 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd810c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdd00726f0 con 0x7fbdd0070ac0 2024-03-16T10:10:36.777 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd810c640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdd00b6f80 con 0x7fbdd0141900 2024-03-16T10:10:36.777 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd810c640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fbdd0072fb0 con 0x7fbdd00b6ba0 2024-03-16T10:10:36.777 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd56a7640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbdd0070ac0 0x7fbdd01411f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:52870/0 (socket says 172.21.15.17:52870) 2024-03-16T10:10:36.777 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd56a7640 1 -- 172.21.15.17:0/1231056109 learned_addr learned my addr 172.21.15.17:0/1231056109 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:10:36.777 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd5ea8640 1 --2- 172.21.15.17:0/1231056109 >> v2:172.21.15.174:3300/0 conn(0x7fbdd00b6ba0 0x7fbdd00721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:36.777 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 -- 172.21.15.17:0/1231056109 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1486002230 0 0) 0x7fbdd00726f0 con 0x7fbdd0070ac0 2024-03-16T10:10:36.777 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 -- 172.21.15.17:0/1231056109 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbdbc003530 con 0x7fbdd0070ac0 2024-03-16T10:10:36.777 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 -- 172.21.15.17:0/1231056109 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3865812682 0 0) 0x7fbdbc003530 con 0x7fbdd0070ac0 2024-03-16T10:10:36.778 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 -- 172.21.15.17:0/1231056109 >> v2:172.21.15.174:3300/0 conn(0x7fbdd00b6ba0 msgr2=0x7fbdd00721b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:36.778 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 --2- 172.21.15.17:0/1231056109 >> v2:172.21.15.174:3300/0 conn(0x7fbdd00b6ba0 0x7fbdd00721b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:36.778 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 -- 172.21.15.17:0/1231056109 >> v1:172.21.15.174:6789/0 conn(0x7fbdd0141900 legacy=0x7fbdd00b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:36.778 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 -- 172.21.15.17:0/1231056109 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd0072960 con 0x7fbdd0070ac0 2024-03-16T10:10:36.778 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 -- 172.21.15.17:0/1231056109 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fbdc0001680 con 0x7fbdd0070ac0 2024-03-16T10:10:36.778 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 -- 172.21.15.17:0/1231056109 >> v1:172.21.15.17:6789/0 conn(0x7fbdd0070ac0 legacy=0x7fbdd01411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:36.778 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 --2- 172.21.15.17:0/1231056109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdbc003a30 0x7fbdbc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:36.778 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 --2- 172.21.15.17:0/1231056109 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbdbc004810 0x7fbdbc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:36.778 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 --2- 172.21.15.17:0/1231056109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc0051d0 0x7fbdbc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:36.778 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 -- 172.21.15.17:0/1231056109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc00c0d0 con 0x7fbdbc003a30 2024-03-16T10:10:36.778 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 -- 172.21.15.17:0/1231056109 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc00c280 con 0x7fbdbc0051d0 2024-03-16T10:10:36.779 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd4ea6640 1 -- 172.21.15.17:0/1231056109 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fbdbc00c480 con 0x7fbdbc004810 2024-03-16T10:10:36.779 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd5ea8640 1 --2- 172.21.15.17:0/1231056109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdbc003a30 0x7fbdbc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:36.779 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd56a7640 1 --2- 172.21.15.17:0/1231056109 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbdbc004810 0x7fbdbc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:36.779 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd66a9640 1 --2- 172.21.15.17:0/1231056109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc0051d0 0x7fbdbc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:36.779 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd5ea8640 1 -- 172.21.15.17:0/1231056109 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbdbc004810 msgr2=0x7fbdbc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:36.779 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd5ea8640 1 --2- 172.21.15.17:0/1231056109 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbdbc004810 0x7fbdbc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:36.779 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd5ea8640 1 -- 172.21.15.17:0/1231056109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc0051d0 msgr2=0x7fbdbc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:36.779 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd5ea8640 1 --2- 172.21.15.17:0/1231056109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc0051d0 0x7fbdbc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:36.779 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd5ea8640 1 -- 172.21.15.17:0/1231056109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdbc003530 con 0x7fbdbc003a30 2024-03-16T10:10:36.779 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd66a9640 1 --2- 172.21.15.17:0/1231056109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc0051d0 0x7fbdbc00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:10:36.779 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd66a9640 1 -- 172.21.15.17:0/1231056109 reap_dead start 2024-03-16T10:10:36.779 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.774+0000 7fbdd56a7640 1 --2- 172.21.15.17:0/1231056109 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbdbc004810 0x7fbdbc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:10:36.783 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd5ea8640 1 --2- 172.21.15.17:0/1231056109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdbc003a30 0x7fbdbc003e50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009ac0 tx=0x7fbdcc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa01bb592799a7c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:36.783 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd4ea6640 1 -- 172.21.15.17:0/1231056109 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc009690 con 0x7fbdbc003a30 2024-03-16T10:10:36.783 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd4ea6640 1 -- 172.21.15.17:0/1231056109 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdcc0a6430 con 0x7fbdbc003a30 2024-03-16T10:10:36.783 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd4ea6640 1 -- 172.21.15.17:0/1231056109 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc0a6800 con 0x7fbdbc003a30 2024-03-16T10:10:36.784 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 -- 172.21.15.17:0/1231056109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdbc003a30 msgr2=0x7fbdbc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:36.784 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 --2- 172.21.15.17:0/1231056109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdbc003a30 0x7fbdbc003e50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc009ac0 tx=0x7fbdcc01ca50 comp rx=0 tx=0).stop 2024-03-16T10:10:36.784 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 -- 172.21.15.17:0/1231056109 shutdown_connections 2024-03-16T10:10:36.784 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 --2- 172.21.15.17:0/1231056109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdbc003a30 0x7fbdbc003e50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:36.784 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 -- 172.21.15.17:0/1231056109 >> 172.21.15.17:0/1231056109 conn(0x7fbdd006c540 msgr2=0x7fbdd006c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:36.784 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 -- 172.21.15.17:0/1231056109 shutdown_connections 2024-03-16T10:10:36.784 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 -- 172.21.15.17:0/1231056109 wait complete. 2024-03-16T10:10:36.784 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 Processor -- start 2024-03-16T10:10:36.784 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 -- start start 2024-03-16T10:10:36.784 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc003a30 0x7fbdd0138340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:36.784 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbdd0138880 0x7fbdd00b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:36.784 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdd00b8730 0x7fbdd01e3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:36.784 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd006b260 con 0x7fbdd00b8730 2024-03-16T10:10:36.785 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd00b5a60 con 0x7fbdbc003a30 2024-03-16T10:10:36.785 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.778+0000 7fbdd810c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd006b3e0 con 0x7fbdd0138880 2024-03-16T10:10:36.785 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd66a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdd00b8730 0x7fbdd01e3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:36.785 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd66a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdd00b8730 0x7fbdd01e3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54296/0 (socket says 172.21.15.17:54296) 2024-03-16T10:10:36.785 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd66a9640 1 -- 172.21.15.17:0/1271034786 learned_addr learned my addr 172.21.15.17:0/1271034786 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:36.785 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd56a7640 1 --2- 172.21.15.17:0/1271034786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbdd0138880 0x7fbdd00b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:36.785 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd5ea8640 1 --2- 172.21.15.17:0/1271034786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc003a30 0x7fbdd0138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:36.785 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd66a9640 1 -- 172.21.15.17:0/1271034786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbdd0138880 msgr2=0x7fbdd00b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:36.785 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd66a9640 1 --2- 172.21.15.17:0/1271034786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbdd0138880 0x7fbdd00b81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:36.785 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd66a9640 1 -- 172.21.15.17:0/1271034786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc003a30 msgr2=0x7fbdd0138340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:36.785 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd66a9640 1 --2- 172.21.15.17:0/1271034786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc003a30 0x7fbdd0138340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:36.785 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd66a9640 1 -- 172.21.15.17:0/1271034786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd00b7a10 con 0x7fbdd00b8730 2024-03-16T10:10:36.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd5ea8640 1 --2- 172.21.15.17:0/1271034786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc003a30 0x7fbdd0138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:10:36.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd56a7640 1 --2- 172.21.15.17:0/1271034786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbdd0138880 0x7fbdd00b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:10:36.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd66a9640 1 --2- 172.21.15.17:0/1271034786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdd00b8730 0x7fbdd01e3950 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00b4bf0 tx=0x7fbdc800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:36.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdc67fc640 1 -- 172.21.15.17:0/1271034786 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8003b80 con 0x7fbdd00b8730 2024-03-16T10:10:36.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdc67fc640 1 -- 172.21.15.17:0/1271034786 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc8017070 con 0x7fbdd00b8730 2024-03-16T10:10:36.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdc67fc640 1 -- 172.21.15.17:0/1271034786 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8005070 con 0x7fbdd00b8730 2024-03-16T10:10:36.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd810c640 1 -- 172.21.15.17:0/1271034786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd00b3df0 con 0x7fbdd00b8730 2024-03-16T10:10:36.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd810c640 1 -- 172.21.15.17:0/1271034786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd014b900 con 0x7fbdd00b8730 2024-03-16T10:10:36.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdc67fc640 1 -- 172.21.15.17:0/1271034786 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbdc8005210 con 0x7fbdd00b8730 2024-03-16T10:10:36.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdc67fc640 1 -- 172.21.15.17:0/1271034786 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fbdc800bc60 con 0x7fbdd00b8730 2024-03-16T10:10:36.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.782+0000 7fbdd810c640 1 -- 172.21.15.17:0/1271034786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdd0069240 con 0x7fbdd00b8730 2024-03-16T10:10:36.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.786+0000 7fbdc67fc640 1 -- 172.21.15.17:0/1271034786 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbdc800b620 con 0x7fbdd00b8730 2024-03-16T10:10:36.843 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.842+0000 7fbdd810c640 1 -- 172.21.15.17:0/1271034786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdd0138c60 con 0x7fbdd00b8730 2024-03-16T10:10:36.844 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.842+0000 7fbdc67fc640 1 -- 172.21.15.17:0/1271034786 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fbdc8005500 con 0x7fbdd00b8730 2024-03-16T10:10:36.844 INFO:teuthology.misc.health.smithi017.stdout: 2024-03-16T10:10:36.844 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":10,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:35.591690+0000","last_up_change":"0.000000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:10:36.845 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.842+0000 7fbdd810c640 1 -- 172.21.15.17:0/1271034786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdd00b8730 msgr2=0x7fbdd01e3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:36.845 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.842+0000 7fbdd810c640 1 --2- 172.21.15.17:0/1271034786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdd00b8730 0x7fbdd01e3950 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00b4bf0 tx=0x7fbdc800dc00 comp rx=0 tx=0).stop 2024-03-16T10:10:36.845 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.842+0000 7fbdd810c640 1 -- 172.21.15.17:0/1271034786 shutdown_connections 2024-03-16T10:10:36.845 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.842+0000 7fbdd810c640 1 --2- 172.21.15.17:0/1271034786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbdd00b8730 0x7fbdd01e3950 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:36.846 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.842+0000 7fbdd810c640 1 --2- 172.21.15.17:0/1271034786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbdd0138880 0x7fbdd00b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:36.846 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.842+0000 7fbdd810c640 1 --2- 172.21.15.17:0/1271034786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc003a30 0x7fbdd0138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:36.846 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.842+0000 7fbdd810c640 1 -- 172.21.15.17:0/1271034786 >> 172.21.15.17:0/1271034786 conn(0x7fbdd006c540 msgr2=0x7fbdd0147500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:36.846 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.842+0000 7fbdd810c640 1 -- 172.21.15.17:0/1271034786 shutdown_connections 2024-03-16T10:10:36.846 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:36.842+0000 7fbdd810c640 1 -- 172.21.15.17:0/1271034786 wait complete. 2024-03-16T10:10:36.859 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-16T10:10:36.912 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:36.910+0000 7faa7f5b9180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T10:10:37.297 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:37.294+0000 7f2adf639180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T10:10:37.372 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:37.370+0000 7f2adf639180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T10:10:37.554 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:37.550+0000 7f2adf639180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T10:10:37.559 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:37.558+0000 7faa7f5b9180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T10:10:37.627 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:37.626+0000 7f2adf639180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T10:10:37.638 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:37.638+0000 7faa7f5b9180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T10:10:37.828 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:37.826+0000 7faa7f5b9180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T10:10:37.905 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:37.902+0000 7faa7f5b9180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T10:10:37.916 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:37.914+0000 7f2adf639180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T10:10:37.977 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:37.974+0000 7f2adf639180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T10:10:38.214 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:38.214+0000 7faa7f5b9180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T10:10:38.279 INFO:tasks.ceph.mgr.y.smithi017.stderr:2024-03-16T10:10:38.278+0000 7faa7f5b9180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T10:10:39.022 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:39.018+0000 7f2a86a81640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-16T10:10:39.023 INFO:tasks.ceph.mgr.x.smithi174.stderr:2024-03-16T10:10:39.022+0000 7f2a86a81640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-16T10:10:40.009 INFO:tasks.ceph.osd.5.smithi174.stderr:2024-03-16T10:10:40.006+0000 7f063160e7c0 -1 osd.5 0 log_to_monitors true 2024-03-16T10:10:40.019 INFO:tasks.ceph.osd.6.smithi174.stderr:2024-03-16T10:10:40.018+0000 7fb6010537c0 -1 osd.6 0 log_to_monitors true 2024-03-16T10:10:40.024 INFO:tasks.ceph.osd.4.smithi174.stderr:2024-03-16T10:10:40.022+0000 7fc3729627c0 -1 osd.4 0 log_to_monitors true 2024-03-16T10:10:40.047 INFO:tasks.ceph.osd.7.smithi174.stderr:2024-03-16T10:10:40.046+0000 7f6f45f3e7c0 -1 osd.7 0 log_to_monitors true 2024-03-16T10:10:40.072 INFO:tasks.ceph.osd.3.smithi017.stderr:2024-03-16T10:10:40.070+0000 7f0649d387c0 -1 osd.3 0 log_to_monitors true 2024-03-16T10:10:40.124 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-03-16T10:10:40.122+0000 7f5f8579c7c0 -1 osd.1 0 log_to_monitors true 2024-03-16T10:10:40.139 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-03-16T10:10:40.138+0000 7fa44e3147c0 -1 osd.2 0 log_to_monitors true 2024-03-16T10:10:40.155 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-03-16T10:10:40.154+0000 7f9c1c9a67c0 -1 osd.0 0 log_to_monitors true 2024-03-16T10:10:43.503 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-03-16T10:10:43.502+0000 7f9c1894f640 -1 osd.0 0 waiting for initial osdmap 2024-03-16T10:10:43.530 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-03-16T10:10:43.530+0000 7f9c10784640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:10:43.548 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-03-16T10:10:43.546+0000 7fa44a2bd640 -1 osd.2 0 waiting for initial osdmap 2024-03-16T10:10:43.570 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-03-16T10:10:43.570+0000 7fa4420f2640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:10:43.576 INFO:tasks.ceph.osd.3.smithi017.stderr:2024-03-16T10:10:43.574+0000 7f0645ce1640 -1 osd.3 0 waiting for initial osdmap 2024-03-16T10:10:43.614 INFO:tasks.ceph.osd.3.smithi017.stderr:2024-03-16T10:10:43.614+0000 7f063db16640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:10:43.617 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-03-16T10:10:43.614+0000 7f5f81745640 -1 osd.1 0 waiting for initial osdmap 2024-03-16T10:10:43.654 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-03-16T10:10:43.654+0000 7f5f7957a640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:10:43.665 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-16T10:10:43.778 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f4406e27640 1 Processor -- start 2024-03-16T10:10:43.778 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f4406e27640 1 -- start start 2024-03-16T10:10:43.779 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f4406e27640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f44000bd6d0 0x7f440014abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:43.780 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f4406e27640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4400141800 0x7f440014b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:43.780 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f4406e27640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44000b6940 con 0x7f44000b9450 2024-03-16T10:10:43.780 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f4406e27640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f44000b5cd0 con 0x7f4400141800 2024-03-16T10:10:43.780 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f4406e27640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f44000b6bb0 con 0x7f44000bd6d0 2024-03-16T10:10:43.780 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f44053c4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4400141800 0x7f440014b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:43.780 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f44053c4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4400141800 0x7f440014b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59578/0 (socket says 172.21.15.17:59578) 2024-03-16T10:10:43.780 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f44053c4640 1 -- 172.21.15.17:0/3708920368 learned_addr learned my addr 172.21.15.17:0/3708920368 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:43.780 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f4404bc3640 1 --2- 172.21.15.17:0/3708920368 >> v2:172.21.15.174:3300/0 conn(0x7f44000bd6d0 0x7f440014abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:43.780 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f44053c4640 1 -- 172.21.15.17:0/3708920368 >> v2:172.21.15.174:3300/0 conn(0x7f44000bd6d0 msgr2=0x7f440014abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:43.780 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f44053c4640 1 --2- 172.21.15.17:0/3708920368 >> v2:172.21.15.174:3300/0 conn(0x7f44000bd6d0 0x7f440014abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:43.780 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f44053c4640 1 -- 172.21.15.17:0/3708920368 >> v1:172.21.15.174:6789/0 conn(0x7f44000b9450 legacy=0x7f44001410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:43.780 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.778+0000 7f44053c4640 1 -- 172.21.15.17:0/3708920368 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44000bdab0 con 0x7f4400141800 2024-03-16T10:10:43.782 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f44053c4640 1 --2- 172.21.15.17:0/3708920368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4400141800 0x7f440014b0f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f44000b8830 tx=0x7f43f801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e568d9b277b9028d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:43.782 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43ff7fe640 1 -- 172.21.15.17:0/3708920368 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f80a5820 con 0x7f4400141800 2024-03-16T10:10:43.783 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43ff7fe640 1 -- 172.21.15.17:0/3708920368 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f80a59c0 con 0x7f4400141800 2024-03-16T10:10:43.783 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43ff7fe640 1 -- 172.21.15.17:0/3708920368 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f80a5d90 con 0x7f4400141800 2024-03-16T10:10:43.783 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 -- 172.21.15.17:0/3708920368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4400141800 msgr2=0x7f440014b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:43.783 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 --2- 172.21.15.17:0/3708920368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4400141800 0x7f440014b0f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f44000b8830 tx=0x7f43f801c960 comp rx=0 tx=0).stop 2024-03-16T10:10:43.783 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 -- 172.21.15.17:0/3708920368 shutdown_connections 2024-03-16T10:10:43.783 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 --2- 172.21.15.17:0/3708920368 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4400141800 0x7f440014b0f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:43.783 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 --2- 172.21.15.17:0/3708920368 >> v2:172.21.15.174:3300/0 conn(0x7f44000bd6d0 0x7f440014abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:43.783 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 -- 172.21.15.17:0/3708920368 >> 172.21.15.17:0/3708920368 conn(0x7f4400072f90 msgr2=0x7f44000bcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:43.783 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 -- 172.21.15.17:0/3708920368 shutdown_connections 2024-03-16T10:10:43.783 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 -- 172.21.15.17:0/3708920368 wait complete. 2024-03-16T10:10:43.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 Processor -- start 2024-03-16T10:10:43.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 -- start start 2024-03-16T10:10:43.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44000b9450 0x7f4400151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:43.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44000bd6d0 0x7f440014fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:43.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4400141800 0x7f440014ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:43.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4400154730 con 0x7f44000bd6d0 2024-03-16T10:10:43.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44001545b0 con 0x7f4400141800 2024-03-16T10:10:43.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f44001548b0 con 0x7f44000b9450 2024-03-16T10:10:43.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43fffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44000bd6d0 0x7f440014fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:43.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43fffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44000bd6d0 0x7f440014fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59592/0 (socket says 172.21.15.17:59592) 2024-03-16T10:10:43.786 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43fffff640 1 -- 172.21.15.17:0/3513154227 learned_addr learned my addr 172.21.15.17:0/3513154227 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:43.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4404bc3640 1 --2- 172.21.15.17:0/3513154227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44000b9450 0x7f4400151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:43.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f44053c4640 1 --2- 172.21.15.17:0/3513154227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4400141800 0x7f440014ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:43.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43fffff640 1 -- 172.21.15.17:0/3513154227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44000b9450 msgr2=0x7f4400151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:43.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43fffff640 1 --2- 172.21.15.17:0/3513154227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44000b9450 0x7f4400151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:43.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43fffff640 1 -- 172.21.15.17:0/3513154227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4400141800 msgr2=0x7f440014ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:43.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43fffff640 1 --2- 172.21.15.17:0/3513154227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4400141800 0x7f440014ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:43.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43fffff640 1 -- 172.21.15.17:0/3513154227 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440014d0f0 con 0x7f44000bd6d0 2024-03-16T10:10:43.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43fffff640 1 --2- 172.21.15.17:0/3513154227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44000bd6d0 0x7f440014fd20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f43f400c420 tx=0x7f43f400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:43.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43fd7fa640 1 -- 172.21.15.17:0/3513154227 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f40052d0 con 0x7f44000bd6d0 2024-03-16T10:10:43.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43fd7fa640 1 -- 172.21.15.17:0/3513154227 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43f4005470 con 0x7f44000bd6d0 2024-03-16T10:10:43.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 -- 172.21.15.17:0/3513154227 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4400150aa0 con 0x7f44000bd6d0 2024-03-16T10:10:43.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f43fd7fa640 1 -- 172.21.15.17:0/3513154227 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f4007260 con 0x7f44000bd6d0 2024-03-16T10:10:43.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.782+0000 7f4406e27640 1 -- 172.21.15.17:0/3513154227 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f440014f0c0 con 0x7f44000bd6d0 2024-03-16T10:10:43.787 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.786+0000 7f4406e27640 1 -- 172.21.15.17:0/3513154227 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4400069240 con 0x7f44000bd6d0 2024-03-16T10:10:43.788 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.786+0000 7f43fd7fa640 1 -- 172.21.15.17:0/3513154227 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 4) ==== 87027+0+0 (secure 0 0 0) 0x7f43f4007400 con 0x7f44000bd6d0 2024-03-16T10:10:43.788 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.786+0000 7f43fd7fa640 1 --2- 172.21.15.17:0/3513154227 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f43ec0675d0 0x7f43ec069a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:43.788 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.786+0000 7f43fd7fa640 1 -- 172.21.15.17:0/3513154227 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f43f4083e50 con 0x7f44000bd6d0 2024-03-16T10:10:43.788 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.786+0000 7f4404bc3640 1 --2- 172.21.15.17:0/3513154227 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f43ec0675d0 0x7f43ec069a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:43.788 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.786+0000 7f4404bc3640 1 --2- 172.21.15.17:0/3513154227 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f43ec0675d0 0x7f43ec069a90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009af0 tx=0x7f43f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:43.791 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.790+0000 7f43fd7fa640 1 -- 172.21.15.17:0/3513154227 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f43f40559d0 con 0x7f44000bd6d0 2024-03-16T10:10:43.946 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.946+0000 7f4406e27640 1 -- 172.21.15.17:0/3513154227 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4400074a30 con 0x7f44000bd6d0 2024-03-16T10:10:43.947 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.946+0000 7f43fd7fa640 1 -- 172.21.15.17:0/3513154227 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f43f40552d0 con 0x7f44000bd6d0 2024-03-16T10:10:43.947 INFO:teuthology.misc.health.smithi017.stdout: 2024-03-16T10:10:43.947 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":12,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:42.021639+0000","last_up_change":"0.000000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:10:43.949 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.946+0000 7f4406e27640 1 -- 172.21.15.17:0/3513154227 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f43ec0675d0 msgr2=0x7f43ec069a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:43.949 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.946+0000 7f4406e27640 1 --2- 172.21.15.17:0/3513154227 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f43ec0675d0 0x7f43ec069a90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f43f0009af0 tx=0x7f43f0008040 comp rx=0 tx=0).stop 2024-03-16T10:10:43.949 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.946+0000 7f4406e27640 1 -- 172.21.15.17:0/3513154227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44000bd6d0 msgr2=0x7f440014fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:43.949 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.946+0000 7f4406e27640 1 --2- 172.21.15.17:0/3513154227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44000bd6d0 0x7f440014fd20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f43f400c420 tx=0x7f43f400c8e0 comp rx=0 tx=0).stop 2024-03-16T10:10:43.949 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.946+0000 7f4406e27640 1 -- 172.21.15.17:0/3513154227 shutdown_connections 2024-03-16T10:10:43.949 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.946+0000 7f4406e27640 1 --2- 172.21.15.17:0/3513154227 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f43ec0675d0 0x7f43ec069a90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:43.949 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.946+0000 7f4406e27640 1 --2- 172.21.15.17:0/3513154227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4400141800 0x7f440014ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:43.949 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.946+0000 7f4406e27640 1 --2- 172.21.15.17:0/3513154227 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44000bd6d0 0x7f440014fd20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:43.949 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.946+0000 7f4406e27640 1 --2- 172.21.15.17:0/3513154227 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44000b9450 0x7f4400151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:43.949 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.946+0000 7f4406e27640 1 -- 172.21.15.17:0/3513154227 >> 172.21.15.17:0/3513154227 conn(0x7f4400072f90 msgr2=0x7f4400147420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:43.950 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.946+0000 7f4406e27640 1 -- 172.21.15.17:0/3513154227 shutdown_connections 2024-03-16T10:10:43.950 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:43.946+0000 7f4406e27640 1 -- 172.21.15.17:0/3513154227 wait complete. 2024-03-16T10:10:43.963 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-16T10:10:44.177 INFO:tasks.ceph.osd.4.smithi174.stderr:2024-03-16T10:10:44.174+0000 7fc36e90b640 -1 osd.4 0 waiting for initial osdmap 2024-03-16T10:10:44.211 INFO:tasks.ceph.osd.4.smithi174.stderr:2024-03-16T10:10:44.210+0000 7fc366740640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:10:44.223 INFO:tasks.ceph.osd.6.smithi174.stderr:2024-03-16T10:10:44.222+0000 7fb5fcffc640 -1 osd.6 0 waiting for initial osdmap 2024-03-16T10:10:44.228 INFO:tasks.ceph.osd.5.smithi174.stderr:2024-03-16T10:10:44.226+0000 7f062d5b7640 -1 osd.5 0 waiting for initial osdmap 2024-03-16T10:10:44.232 INFO:tasks.ceph.osd.7.smithi174.stderr:2024-03-16T10:10:44.230+0000 7f6f41ee7640 -1 osd.7 0 waiting for initial osdmap 2024-03-16T10:10:44.251 INFO:tasks.ceph.osd.5.smithi174.stderr:2024-03-16T10:10:44.250+0000 7f06253ec640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:10:44.254 INFO:tasks.ceph.osd.6.smithi174.stderr:2024-03-16T10:10:44.254+0000 7fb5f4e31640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:10:44.263 INFO:tasks.ceph.osd.7.smithi174.stderr:2024-03-16T10:10:44.262+0000 7f6f39d1c640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T10:10:50.770 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-16T10:10:50.902 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.898+0000 7fde344c5640 1 Processor -- start 2024-03-16T10:10:50.902 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.898+0000 7fde344c5640 1 -- start start 2024-03-16T10:10:50.904 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde344c5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fde2c141900 0x7fde2c14ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:50.904 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde344c5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fde2c0b9450 0x7fde2c14b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:50.904 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde344c5640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde2c0bdab0 con 0x7fde2c0bd6d0 2024-03-16T10:10:50.904 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde344c5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fde2c0b5d40 con 0x7fde2c0b9450 2024-03-16T10:10:50.904 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde344c5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fde2c0b56f0 con 0x7fde2c141900 2024-03-16T10:10:50.904 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde32a62640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fde2c0b9450 0x7fde2c14b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:50.904 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde32a62640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fde2c0b9450 0x7fde2c14b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43480/0 (socket says 172.21.15.17:43480) 2024-03-16T10:10:50.904 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde32a62640 1 -- 172.21.15.17:0/3980716039 learned_addr learned my addr 172.21.15.17:0/3980716039 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:50.904 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde31a60640 1 --2- 172.21.15.17:0/3980716039 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fde2c141900 0x7fde2c14ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:50.904 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde32a62640 1 -- 172.21.15.17:0/3980716039 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fde2c141900 msgr2=0x7fde2c14ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:50.905 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde32a62640 1 --2- 172.21.15.17:0/3980716039 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fde2c141900 0x7fde2c14ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:50.905 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde32a62640 1 -- 172.21.15.17:0/3980716039 >> v1:172.21.15.174:6789/0 conn(0x7fde2c0bd6d0 legacy=0x7fde2c1411f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:50.905 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde32a62640 1 -- 172.21.15.17:0/3980716039 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2c0b59d0 con 0x7fde2c0b9450 2024-03-16T10:10:50.906 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde32a62640 1 --2- 172.21.15.17:0/3980716039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c0b9450 0x7fde2c14b160 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fde24009870 tx=0x7fde2401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c663994cd03fd199 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:50.906 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde3125f640 1 -- 172.21.15.17:0/3980716039 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde240b2070 con 0x7fde2c0b9450 2024-03-16T10:10:50.906 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde3125f640 1 -- 172.21.15.17:0/3980716039 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde240a9070 con 0x7fde2c0b9450 2024-03-16T10:10:50.907 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.902+0000 7fde3125f640 1 -- 172.21.15.17:0/3980716039 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde240ad880 con 0x7fde2c0b9450 2024-03-16T10:10:50.907 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 -- 172.21.15.17:0/3980716039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c0b9450 msgr2=0x7fde2c14b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:50.908 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 --2- 172.21.15.17:0/3980716039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c0b9450 0x7fde2c14b160 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fde24009870 tx=0x7fde2401c490 comp rx=0 tx=0).stop 2024-03-16T10:10:50.908 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 -- 172.21.15.17:0/3980716039 shutdown_connections 2024-03-16T10:10:50.908 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 --2- 172.21.15.17:0/3980716039 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c0b9450 0x7fde2c14b160 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:50.908 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 --2- 172.21.15.17:0/3980716039 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fde2c141900 0x7fde2c14ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:50.908 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 -- 172.21.15.17:0/3980716039 >> 172.21.15.17:0/3980716039 conn(0x7fde2c072f90 msgr2=0x7fde2c0bcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:50.908 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 -- 172.21.15.17:0/3980716039 shutdown_connections 2024-03-16T10:10:50.908 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 -- 172.21.15.17:0/3980716039 wait complete. 2024-03-16T10:10:50.908 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 Processor -- start 2024-03-16T10:10:50.909 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 -- start start 2024-03-16T10:10:50.911 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fde2c0b9450 0x7fde2c138400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:50.911 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c0bd6d0 0x7fde2c1dfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:50.911 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde2c141900 0x7fde2c133df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:50.911 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c154920 con 0x7fde2c0bd6d0 2024-03-16T10:10:50.911 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c1547a0 con 0x7fde2c141900 2024-03-16T10:10:50.911 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde344c5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fde2c154aa0 con 0x7fde2c0b9450 2024-03-16T10:10:50.911 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde31a60640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c0bd6d0 0x7fde2c1dfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:50.911 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde31a60640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c0bd6d0 0x7fde2c1dfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43488/0 (socket says 172.21.15.17:43488) 2024-03-16T10:10:50.911 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde31a60640 1 -- 172.21.15.17:0/366551584 learned_addr learned my addr 172.21.15.17:0/366551584 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:50.911 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde32261640 1 --2- 172.21.15.17:0/366551584 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fde2c0b9450 0x7fde2c138400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:50.911 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde32a62640 1 --2- 172.21.15.17:0/366551584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde2c141900 0x7fde2c133df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:50.911 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde31a60640 1 -- 172.21.15.17:0/366551584 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fde2c0b9450 msgr2=0x7fde2c138400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:50.911 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde31a60640 1 --2- 172.21.15.17:0/366551584 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fde2c0b9450 0x7fde2c138400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:50.911 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde31a60640 1 -- 172.21.15.17:0/366551584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde2c141900 msgr2=0x7fde2c133df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:50.912 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde31a60640 1 --2- 172.21.15.17:0/366551584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde2c141900 0x7fde2c133df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:50.912 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde31a60640 1 -- 172.21.15.17:0/366551584 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2c13a0b0 con 0x7fde2c0bd6d0 2024-03-16T10:10:50.912 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.906+0000 7fde31a60640 1 --2- 172.21.15.17:0/366551584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c0bd6d0 0x7fde2c1dfdf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fde20007e90 tx=0x7fde2000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:50.912 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.910+0000 7fde2affd640 1 -- 172.21.15.17:0/366551584 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20014620 con 0x7fde2c0bd6d0 2024-03-16T10:10:50.912 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.910+0000 7fde2affd640 1 -- 172.21.15.17:0/366551584 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde20019070 con 0x7fde2c0bd6d0 2024-03-16T10:10:50.912 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.910+0000 7fde2affd640 1 -- 172.21.15.17:0/366551584 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20014a20 con 0x7fde2c0bd6d0 2024-03-16T10:10:50.912 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.910+0000 7fde344c5640 1 -- 172.21.15.17:0/366551584 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2c0bc8a0 con 0x7fde2c0bd6d0 2024-03-16T10:10:50.912 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.910+0000 7fde344c5640 1 -- 172.21.15.17:0/366551584 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde2c139940 con 0x7fde2c0bd6d0 2024-03-16T10:10:50.913 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.910+0000 7fde344c5640 1 -- 172.21.15.17:0/366551584 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde2c069240 con 0x7fde2c0bd6d0 2024-03-16T10:10:50.913 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.910+0000 7fde2affd640 1 -- 172.21.15.17:0/366551584 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fde20014bc0 con 0x7fde2c0bd6d0 2024-03-16T10:10:50.913 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.910+0000 7fde2affd640 1 --2- 172.21.15.17:0/366551584 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fde18067660 0x7fde18069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:50.913 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.910+0000 7fde2affd640 1 -- 172.21.15.17:0/366551584 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fde20085dc0 con 0x7fde2c0bd6d0 2024-03-16T10:10:50.913 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.910+0000 7fde32261640 1 --2- 172.21.15.17:0/366551584 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fde18067660 0x7fde18069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:50.913 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.910+0000 7fde32261640 1 --2- 172.21.15.17:0/366551584 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fde18067660 0x7fde18069b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fde2c14a460 tx=0x7fde1c007e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:50.917 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:50.914+0000 7fde2affd640 1 -- 172.21.15.17:0/366551584 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fde20056d80 con 0x7fde2c0bd6d0 2024-03-16T10:10:51.078 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:51.074+0000 7fde344c5640 1 -- 172.21.15.17:0/366551584 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde2c0b65f0 con 0x7fde2c0bd6d0 2024-03-16T10:10:51.080 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:51.078+0000 7fde2affd640 1 -- 172.21.15.17:0/366551584 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12884 (secure 0 0 0) 0x7fde20056680 con 0x7fde2c0bd6d0 2024-03-16T10:10:51.080 INFO:teuthology.misc.health.smithi017.stdout: 2024-03-16T10:10:51.081 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":17,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:48.527781+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:10:51.082 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:51.078+0000 7fde344c5640 1 -- 172.21.15.17:0/366551584 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fde18067660 msgr2=0x7fde18069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.082 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:51.078+0000 7fde344c5640 1 --2- 172.21.15.17:0/366551584 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fde18067660 0x7fde18069b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fde2c14a460 tx=0x7fde1c007e40 comp rx=0 tx=0).stop 2024-03-16T10:10:51.083 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:51.078+0000 7fde344c5640 1 -- 172.21.15.17:0/366551584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c0bd6d0 msgr2=0x7fde2c1dfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.083 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:51.078+0000 7fde344c5640 1 --2- 172.21.15.17:0/366551584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c0bd6d0 0x7fde2c1dfdf0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fde20007e90 tx=0x7fde2000e4b0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.083 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:51.082+0000 7fde344c5640 1 -- 172.21.15.17:0/366551584 shutdown_connections 2024-03-16T10:10:51.083 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:51.082+0000 7fde344c5640 1 --2- 172.21.15.17:0/366551584 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fde18067660 0x7fde18069b20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.083 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:51.082+0000 7fde344c5640 1 --2- 172.21.15.17:0/366551584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde2c141900 0x7fde2c133df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.083 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:51.082+0000 7fde344c5640 1 --2- 172.21.15.17:0/366551584 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde2c0bd6d0 0x7fde2c1dfdf0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.083 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:51.082+0000 7fde344c5640 1 --2- 172.21.15.17:0/366551584 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fde2c0b9450 0x7fde2c138400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.083 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:51.082+0000 7fde344c5640 1 -- 172.21.15.17:0/366551584 >> 172.21.15.17:0/366551584 conn(0x7fde2c072f90 msgr2=0x7fde2c13f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:51.083 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:51.082+0000 7fde344c5640 1 -- 172.21.15.17:0/366551584 shutdown_connections 2024-03-16T10:10:51.083 INFO:teuthology.misc.health.smithi017.stderr:2024-03-16T10:10:51.082+0000 7fde344c5640 1 -- 172.21.15.17:0/366551584 wait complete. 2024-03-16T10:10:51.098 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-16T10:10:51.098 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-16T10:10:51.098 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-16T10:10:51.098 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-16T10:10:51.101 INFO:tasks.ceph.mds.a:Started 2024-03-16T10:10:51.101 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-16T10:10:51.101 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-16T10:10:51.105 INFO:tasks.ceph.mds.b:Started 2024-03-16T10:10:51.105 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-16T10:10:51.105 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:10:51.183 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-16T10:10:51.203 INFO:tasks.ceph.mds.b.smithi174.stdout:starting mds.b at 2024-03-16T10:10:51.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f286e3df640 1 Processor -- start 2024-03-16T10:10:51.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f286e3df640 1 -- start start 2024-03-16T10:10:51.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f286e3df640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f28680b7170 0x7f286806f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f286e3df640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28681487a0 0x7f286814fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f286e3df640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28680730a0 con 0x7f2868072cc0 2024-03-16T10:10:51.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f286e3df640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f286806eee0 con 0x7f28680b7170 2024-03-16T10:10:51.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f286e3df640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2868148c50 con 0x7f28681487a0 2024-03-16T10:10:51.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f2867fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2868072cc0 0x7f286813f150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47476/0 (socket says 172.21.15.17:47476) 2024-03-16T10:10:51.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f2867fff640 1 -- 172.21.15.17:0/2109817654 learned_addr learned my addr 172.21.15.17:0/2109817654 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:10:51.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f28677fe640 1 --2- 172.21.15.17:0/2109817654 >> v2:172.21.15.174:3300/0 conn(0x7f28680b7170 0x7f286806f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f286c97c640 1 --2- 172.21.15.17:0/2109817654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28681487a0 0x7f286814fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f2866ffd640 1 -- 172.21.15.17:0/2109817654 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2529343606 0 0) 0x7f28680730a0 con 0x7f2868072cc0 2024-03-16T10:10:51.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f2866ffd640 1 -- 172.21.15.17:0/2109817654 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2850003530 con 0x7f2868072cc0 2024-03-16T10:10:51.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f28677fe640 1 -- 172.21.15.17:0/2109817654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28681487a0 msgr2=0x7f286814fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:51.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f28677fe640 1 --2- 172.21.15.17:0/2109817654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28681487a0 0x7f286814fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f28677fe640 1 -- 172.21.15.17:0/2109817654 >> v1:172.21.15.17:6789/0 conn(0x7f2868072cc0 legacy=0x7f286813f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.222+0000 7f28677fe640 1 -- 172.21.15.17:0/2109817654 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28680b02e0 con 0x7f28680b7170 2024-03-16T10:10:51.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.226+0000 7f28677fe640 1 --2- 172.21.15.17:0/2109817654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28680b7170 0x7f286806f370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f285c0097c0 tx=0x7f285c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4558e33287f5d1d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:51.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.226+0000 7f2866ffd640 1 -- 172.21.15.17:0/2109817654 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c0b1070 con 0x7f28680b7170 2024-03-16T10:10:51.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.226+0000 7f2866ffd640 1 -- 172.21.15.17:0/2109817654 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f285c0a8050 con 0x7f28680b7170 2024-03-16T10:10:51.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.226+0000 7f2866ffd640 1 -- 172.21.15.17:0/2109817654 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c0ac860 con 0x7f28680b7170 2024-03-16T10:10:51.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.226+0000 7f286e3df640 1 -- 172.21.15.17:0/2109817654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28680b7170 msgr2=0x7f286806f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.226+0000 7f286e3df640 1 --2- 172.21.15.17:0/2109817654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28680b7170 0x7f286806f370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f285c0097c0 tx=0x7f285c01c3e0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.226+0000 7f286e3df640 1 -- 172.21.15.17:0/2109817654 shutdown_connections 2024-03-16T10:10:51.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.226+0000 7f286e3df640 1 --2- 172.21.15.17:0/2109817654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28681487a0 0x7f286814fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.226+0000 7f286e3df640 1 --2- 172.21.15.17:0/2109817654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28680b7170 0x7f286806f370 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.226+0000 7f286e3df640 1 -- 172.21.15.17:0/2109817654 >> 172.21.15.17:0/2109817654 conn(0x7f28680b7960 msgr2=0x7f28680b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:51.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.226+0000 7f286e3df640 1 -- 172.21.15.17:0/2109817654 shutdown_connections 2024-03-16T10:10:51.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.226+0000 7f286e3df640 1 -- 172.21.15.17:0/2109817654 wait complete. 2024-03-16T10:10:51.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.226+0000 7f286e3df640 1 Processor -- start 2024-03-16T10:10:51.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.226+0000 7f286e3df640 1 -- start start 2024-03-16T10:10:51.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286e3df640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2868072cc0 0x7f28681450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286e3df640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28680b7170 0x7f28681ddc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286e3df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28681487a0 0x7f2868134fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286e3df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f286806ff40 con 0x7f28681487a0 2024-03-16T10:10:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286e3df640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f286806fdc0 con 0x7f2868072cc0 2024-03-16T10:10:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286e3df640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f286813f860 con 0x7f28680b7170 2024-03-16T10:10:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286c97c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28681487a0 0x7f2868134fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286c97c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28681487a0 0x7f2868134fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43524/0 (socket says 172.21.15.17:43524) 2024-03-16T10:10:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286c97c640 1 -- 172.21.15.17:0/1958033913 learned_addr learned my addr 172.21.15.17:0/1958033913 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f28677fe640 1 --2- 172.21.15.17:0/1958033913 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28680b7170 0x7f28681ddc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f2867fff640 1 --2- 172.21.15.17:0/1958033913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2868072cc0 0x7f28681450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286c97c640 1 -- 172.21.15.17:0/1958033913 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28680b7170 msgr2=0x7f28681ddc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286c97c640 1 --2- 172.21.15.17:0/1958033913 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28680b7170 0x7f28681ddc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286c97c640 1 -- 172.21.15.17:0/1958033913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2868072cc0 msgr2=0x7f28681450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286c97c640 1 --2- 172.21.15.17:0/1958033913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2868072cc0 0x7f28681450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286c97c640 1 -- 172.21.15.17:0/1958033913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286813b960 con 0x7f28681487a0 2024-03-16T10:10:51.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286c97c640 1 --2- 172.21.15.17:0/1958033913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28681487a0 0x7f2868134fb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f286000efc0 tx=0x7f286000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:51.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f2864ff9640 1 -- 172.21.15.17:0/1958033913 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860018020 con 0x7f28681487a0 2024-03-16T10:10:51.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f2864ff9640 1 -- 172.21.15.17:0/1958033913 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2860014480 con 0x7f28681487a0 2024-03-16T10:10:51.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f2864ff9640 1 -- 172.21.15.17:0/1958033913 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860014870 con 0x7f28681487a0 2024-03-16T10:10:51.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286e3df640 1 -- 172.21.15.17:0/1958033913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28680b2390 con 0x7f28681487a0 2024-03-16T10:10:51.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286e3df640 1 -- 172.21.15.17:0/1958033913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286813b4d0 con 0x7f28681487a0 2024-03-16T10:10:51.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f286e3df640 1 -- 172.21.15.17:0/1958033913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28680b2560 con 0x7f28681487a0 2024-03-16T10:10:51.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f2864ff9640 1 -- 172.21.15.17:0/1958033913 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f286001b020 con 0x7f28681487a0 2024-03-16T10:10:51.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f2864ff9640 1 --2- 172.21.15.17:0/1958033913 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2854067660 0x7f2854069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f2864ff9640 1 -- 172.21.15.17:0/1958033913 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f2860085e10 con 0x7f28681487a0 2024-03-16T10:10:51.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.230+0000 7f2867fff640 1 --2- 172.21.15.17:0/1958033913 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2854067660 0x7f2854069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.234+0000 7f2867fff640 1 --2- 172.21.15.17:0/1958033913 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2854067660 0x7f2854069b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2858007fd0 tx=0x7f2858009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:51.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.238+0000 7f2864ff9640 1 -- 172.21.15.17:0/1958033913 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2860022070 con 0x7f28681487a0 2024-03-16T10:10:51.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.398+0000 7f286e3df640 1 -- 172.21.15.17:0/1958033913 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28680b5860 con 0x7f28681487a0 2024-03-16T10:10:51.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.398+0000 7f2864ff9640 1 -- 172.21.15.17:0/1958033913 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12884 (secure 0 0 0) 0x7f2860010040 con 0x7f28681487a0 2024-03-16T10:10:51.401 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:51.401 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:48.527781+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:10:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.402+0000 7f286e3df640 1 -- 172.21.15.17:0/1958033913 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2854067660 msgr2=0x7f2854069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.402+0000 7f286e3df640 1 --2- 172.21.15.17:0/1958033913 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2854067660 0x7f2854069b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2858007fd0 tx=0x7f2858009040 comp rx=0 tx=0).stop 2024-03-16T10:10:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.402+0000 7f286e3df640 1 -- 172.21.15.17:0/1958033913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28681487a0 msgr2=0x7f2868134fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.402+0000 7f286e3df640 1 --2- 172.21.15.17:0/1958033913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28681487a0 0x7f2868134fb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f286000efc0 tx=0x7f286000c490 comp rx=0 tx=0).stop 2024-03-16T10:10:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.402+0000 7f286e3df640 1 -- 172.21.15.17:0/1958033913 shutdown_connections 2024-03-16T10:10:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.402+0000 7f286e3df640 1 --2- 172.21.15.17:0/1958033913 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2854067660 0x7f2854069b20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.402+0000 7f286e3df640 1 --2- 172.21.15.17:0/1958033913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28681487a0 0x7f2868134fb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.402+0000 7f286e3df640 1 --2- 172.21.15.17:0/1958033913 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28680b7170 0x7f28681ddc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.402+0000 7f286e3df640 1 --2- 172.21.15.17:0/1958033913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2868072cc0 0x7f28681450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.402+0000 7f286e3df640 1 -- 172.21.15.17:0/1958033913 >> 172.21.15.17:0/1958033913 conn(0x7f28680b7960 msgr2=0x7f286813ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.402+0000 7f286e3df640 1 -- 172.21.15.17:0/1958033913 shutdown_connections 2024-03-16T10:10:51.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.402+0000 7f286e3df640 1 -- 172.21.15.17:0/1958033913 wait complete. 2024-03-16T10:10:51.422 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:10:51.422 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:10:51.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c2849c640 1 Processor -- start 2024-03-16T10:10:51.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c2849c640 1 -- start start 2024-03-16T10:10:51.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c2849c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3c200b4800 0x7f3c200b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c2849c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3c20147ca0 0x7f3c201480d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c2849c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c200b1ac0 con 0x7f3c200b1f70 2024-03-16T10:10:51.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c2849c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3c200b4be0 con 0x7f3c20147ca0 2024-03-16T10:10:51.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c2849c640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f3c200b1220 con 0x7f3c200b4800 2024-03-16T10:10:51.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c26a39640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3c20147ca0 0x7f3c201480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c25a37640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3c200b1f70 0x7f3c200b23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47486/0 (socket says 172.21.15.17:47486) 2024-03-16T10:10:51.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c25a37640 1 -- 172.21.15.17:0/803383042 learned_addr learned my addr 172.21.15.17:0/803383042 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:10:51.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c26238640 1 --2- 172.21.15.17:0/803383042 >> v2:172.21.15.174:3300/0 conn(0x7f3c200b4800 0x7f3c200b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c25236640 1 -- 172.21.15.17:0/803383042 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 930712150 0 0) 0x7f3c200b1ac0 con 0x7f3c200b1f70 2024-03-16T10:10:51.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c25236640 1 -- 172.21.15.17:0/803383042 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c0c003530 con 0x7f3c200b1f70 2024-03-16T10:10:51.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c26a39640 1 -- 172.21.15.17:0/803383042 >> v2:172.21.15.174:3300/0 conn(0x7f3c200b4800 msgr2=0x7f3c200b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:51.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c26a39640 1 --2- 172.21.15.17:0/803383042 >> v2:172.21.15.174:3300/0 conn(0x7f3c200b4800 0x7f3c200b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c26a39640 1 -- 172.21.15.17:0/803383042 >> v1:172.21.15.17:6789/0 conn(0x7f3c200b1f70 legacy=0x7f3c200b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.542+0000 7f3c26a39640 1 -- 172.21.15.17:0/803383042 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c200b1500 con 0x7f3c20147ca0 2024-03-16T10:10:51.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c26a39640 1 --2- 172.21.15.17:0/803383042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c20147ca0 0x7f3c201480d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3c200adf40 tx=0x7f3c1801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5197ff0fdd7644eb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:51.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c25236640 1 -- 172.21.15.17:0/803383042 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c180b2070 con 0x7f3c20147ca0 2024-03-16T10:10:51.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c25236640 1 -- 172.21.15.17:0/803383042 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c180a5870 con 0x7f3c20147ca0 2024-03-16T10:10:51.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c25236640 1 -- 172.21.15.17:0/803383042 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c180a5c40 con 0x7f3c20147ca0 2024-03-16T10:10:51.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 -- 172.21.15.17:0/803383042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c20147ca0 msgr2=0x7f3c201480d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 --2- 172.21.15.17:0/803383042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c20147ca0 0x7f3c201480d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3c200adf40 tx=0x7f3c1801c8a0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 -- 172.21.15.17:0/803383042 shutdown_connections 2024-03-16T10:10:51.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 --2- 172.21.15.17:0/803383042 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c20147ca0 0x7f3c201480d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 --2- 172.21.15.17:0/803383042 >> v2:172.21.15.174:3300/0 conn(0x7f3c200b4800 0x7f3c200b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 -- 172.21.15.17:0/803383042 >> 172.21.15.17:0/803383042 conn(0x7f3c200b8e90 msgr2=0x7f3c200b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:51.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 -- 172.21.15.17:0/803383042 shutdown_connections 2024-03-16T10:10:51.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 -- 172.21.15.17:0/803383042 wait complete. 2024-03-16T10:10:51.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 Processor -- start 2024-03-16T10:10:51.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 -- start start 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c200b1f70 0x7f3c200b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c200b4800 0x7f3c20142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c20147ca0 0x7f3c20142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3c201515b0 con 0x7f3c200b1f70 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c20151430 con 0x7f3c200b4800 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c2849c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3c20151730 con 0x7f3c20147ca0 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c26238640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c200b1f70 0x7f3c200b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c26238640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c200b1f70 0x7f3c200b36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43552/0 (socket says 172.21.15.17:43552) 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c26238640 1 -- 172.21.15.17:0/1161424853 learned_addr learned my addr 172.21.15.17:0/1161424853 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c25a37640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c200b4800 0x7f3c20142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c26a39640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c20147ca0 0x7f3c20142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c26238640 1 -- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c20147ca0 msgr2=0x7f3c20142d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c26238640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c20147ca0 0x7f3c20142d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c26238640 1 -- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c200b4800 msgr2=0x7f3c20142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c26238640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c200b4800 0x7f3c20142840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c26238640 1 -- 172.21.15.17:0/1161424853 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c201421a0 con 0x7f3c200b1f70 2024-03-16T10:10:51.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c25a37640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c200b4800 0x7f3c20142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:10:51.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c26238640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c200b1f70 0x7f3c200b36f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00d800 tx=0x7f3c1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:51.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.546+0000 7f3c16ffd640 1 -- 172.21.15.17:0/1161424853 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c00b6c0 con 0x7f3c200b1f70 2024-03-16T10:10:51.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.550+0000 7f3c16ffd640 1 -- 172.21.15.17:0/1161424853 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1c019070 con 0x7f3c200b1f70 2024-03-16T10:10:51.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.550+0000 7f3c2849c640 1 -- 172.21.15.17:0/1161424853 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c200b4090 con 0x7f3c200b1f70 2024-03-16T10:10:51.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.550+0000 7f3c16ffd640 1 -- 172.21.15.17:0/1161424853 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c00ba60 con 0x7f3c200b1f70 2024-03-16T10:10:51.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.550+0000 7f3c2849c640 1 -- 172.21.15.17:0/1161424853 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c20141c10 con 0x7f3c200b1f70 2024-03-16T10:10:51.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.550+0000 7f3c2849c640 1 -- 172.21.15.17:0/1161424853 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c20068360 con 0x7f3c200b1f70 2024-03-16T10:10:51.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.550+0000 7f3c16ffd640 1 -- 172.21.15.17:0/1161424853 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f3c1c00bc00 con 0x7f3c200b1f70 2024-03-16T10:10:51.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.550+0000 7f3c16ffd640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3c04067660 0x7f3c04069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.550+0000 7f3c16ffd640 1 -- 172.21.15.17:0/1161424853 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f3c1c085a30 con 0x7f3c200b1f70 2024-03-16T10:10:51.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.550+0000 7f3c25a37640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3c04067660 0x7f3c04069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.550+0000 7f3c25a37640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3c04067660 0x7f3c04069b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3c201db8a0 tx=0x7f3c10009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:51.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.554+0000 7f3c16ffd640 1 -- 172.21.15.17:0/1161424853 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3c1c0569f0 con 0x7f3c200b1f70 2024-03-16T10:10:51.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.714+0000 7f3c2849c640 1 -- 172.21.15.17:0/1161424853 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3c200af190 con 0x7f3c200b1f70 2024-03-16T10:10:51.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.714+0000 7f3c16ffd640 1 -- 172.21.15.17:0/1161424853 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f3c1c0562f0 con 0x7f3c200b1f70 2024-03-16T10:10:51.718 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:10:51.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.718+0000 7f3c2849c640 1 -- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3c04067660 msgr2=0x7f3c04069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.718+0000 7f3c2849c640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3c04067660 0x7f3c04069b20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3c201db8a0 tx=0x7f3c10009040 comp rx=0 tx=0).stop 2024-03-16T10:10:51.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.718+0000 7f3c2849c640 1 -- 172.21.15.17:0/1161424853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c200b1f70 msgr2=0x7f3c200b36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.718+0000 7f3c2849c640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c200b1f70 0x7f3c200b36f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00d800 tx=0x7f3c1c00dcc0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.718+0000 7f3c2849c640 1 -- 172.21.15.17:0/1161424853 shutdown_connections 2024-03-16T10:10:51.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.718+0000 7f3c2849c640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3c04067660 0x7f3c04069b20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.718+0000 7f3c2849c640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3c20147ca0 0x7f3c20142d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.718+0000 7f3c2849c640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c200b4800 0x7f3c20142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.718+0000 7f3c2849c640 1 --2- 172.21.15.17:0/1161424853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c200b1f70 0x7f3c200b36f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.718+0000 7f3c2849c640 1 -- 172.21.15.17:0/1161424853 >> 172.21.15.17:0/1161424853 conn(0x7f3c200b8e90 msgr2=0x7f3c2006cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:51.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.718+0000 7f3c2849c640 1 -- 172.21.15.17:0/1161424853 shutdown_connections 2024-03-16T10:10:51.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.718+0000 7f3c2849c640 1 -- 172.21.15.17:0/1161424853 wait complete. 2024-03-16T10:10:51.737 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:10:51.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7f5246640 1 Processor -- start 2024-03-16T10:10:51.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7f5246640 1 -- start start 2024-03-16T10:10:51.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7f5246640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd7f0145900 0x7fd7f0145d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7f5246640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7f013f1a0 0x7fd7f013f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7f5246640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7f0070290 con 0x7fd7f0072b80 2024-03-16T10:10:51.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7f5246640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f0070570 con 0x7fd7f0145900 2024-03-16T10:10:51.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7f5246640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f0145440 con 0x7fd7f013f1a0 2024-03-16T10:10:51.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7ef59e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7f013f1a0 0x7fd7f013f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7e7fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd7f0145900 0x7fd7f0145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7e7fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd7f0145900 0x7fd7f0145d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.17:57918/0 (socket says 172.21.15.17:57918) 2024-03-16T10:10:51.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7ef59e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7f013f1a0 0x7fd7f013f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.17:41944/0 (socket says 172.21.15.17:41944) 2024-03-16T10:10:51.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7e7fff640 1 -- 172.21.15.17:0/2817941510 learned_addr learned my addr 172.21.15.17:0/2817941510 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:51.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7ef59e640 1 -- 172.21.15.17:0/2817941510 >> v2:172.21.15.174:3300/0 conn(0x7fd7f0145900 msgr2=0x7fd7f0145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:51.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7ef59e640 1 --2- 172.21.15.17:0/2817941510 >> v2:172.21.15.174:3300/0 conn(0x7fd7f0145900 0x7fd7f0145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7ef59e640 1 -- 172.21.15.17:0/2817941510 >> v1:172.21.15.174:6789/0 conn(0x7fd7f0072b80 legacy=0x7fd7f00b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7ef59e640 1 -- 172.21.15.17:0/2817941510 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f01462c0 con 0x7fd7f013f1a0 2024-03-16T10:10:51.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.854+0000 7fd7e7fff640 1 --2- 172.21.15.17:0/2817941510 >> v2:172.21.15.174:3300/0 conn(0x7fd7f0145900 0x7fd7f0145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:10:51.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.858+0000 7fd7ef59e640 1 --2- 172.21.15.17:0/2817941510 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7f013f1a0 0x7fd7f013f5f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009d90 tx=0x7fd7e801c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b184e31bd0a3bd60 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:51.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.858+0000 7fd7ee59c640 1 -- 172.21.15.17:0/2817941510 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80b2070 con 0x7fd7f013f1a0 2024-03-16T10:10:51.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.858+0000 7fd7ee59c640 1 -- 172.21.15.17:0/2817941510 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e80a5800 con 0x7fd7f013f1a0 2024-03-16T10:10:51.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.858+0000 7fd7ee59c640 1 -- 172.21.15.17:0/2817941510 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80a5bd0 con 0x7fd7f013f1a0 2024-03-16T10:10:51.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.858+0000 7fd7f5246640 1 -- 172.21.15.17:0/2817941510 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7f013f1a0 msgr2=0x7fd7f013f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.858+0000 7fd7f5246640 1 --2- 172.21.15.17:0/2817941510 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7f013f1a0 0x7fd7f013f5f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009d90 tx=0x7fd7e801c850 comp rx=0 tx=0).stop 2024-03-16T10:10:51.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.858+0000 7fd7f5246640 1 -- 172.21.15.17:0/2817941510 shutdown_connections 2024-03-16T10:10:51.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.858+0000 7fd7f5246640 1 --2- 172.21.15.17:0/2817941510 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7f013f1a0 0x7fd7f013f5f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.858+0000 7fd7f5246640 1 --2- 172.21.15.17:0/2817941510 >> v2:172.21.15.174:3300/0 conn(0x7fd7f0145900 0x7fd7f0145d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.858+0000 7fd7f5246640 1 -- 172.21.15.17:0/2817941510 >> 172.21.15.17:0/2817941510 conn(0x7fd7f00b7880 msgr2=0x7fd7f00b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:51.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.858+0000 7fd7f5246640 1 -- 172.21.15.17:0/2817941510 shutdown_connections 2024-03-16T10:10:51.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.858+0000 7fd7f5246640 1 -- 172.21.15.17:0/2817941510 wait complete. 2024-03-16T10:10:51.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7f5246640 1 Processor -- start 2024-03-16T10:10:51.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7f5246640 1 -- start start 2024-03-16T10:10:51.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7f5246640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7f0072b80 0x7fd7f0140b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7f5246640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7f013f1a0 0x7fd7f013e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7f5246640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f0145900 0x7fd7f01dd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7f5246640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f0152500 con 0x7fd7f0145900 2024-03-16T10:10:51.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7f5246640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f013fb30 con 0x7fd7f0072b80 2024-03-16T10:10:51.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7f5246640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f0152680 con 0x7fd7f013f1a0 2024-03-16T10:10:51.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7ef59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f0145900 0x7fd7f01dd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7eed9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7f0072b80 0x7fd7f0140b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7ef59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f0145900 0x7fd7f01dd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43566/0 (socket says 172.21.15.17:43566) 2024-03-16T10:10:51.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7ef59e640 1 -- 172.21.15.17:0/2175476759 learned_addr learned my addr 172.21.15.17:0/2175476759 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:51.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7e7fff640 1 --2- 172.21.15.17:0/2175476759 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7f013f1a0 0x7fd7f013e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7ef59e640 1 -- 172.21.15.17:0/2175476759 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7f013f1a0 msgr2=0x7fd7f013e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7ef59e640 1 --2- 172.21.15.17:0/2175476759 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7f013f1a0 0x7fd7f013e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7ef59e640 1 -- 172.21.15.17:0/2175476759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7f0072b80 msgr2=0x7fd7f0140b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:51.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7ef59e640 1 --2- 172.21.15.17:0/2175476759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7f0072b80 0x7fd7f0140b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:51.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7ef59e640 1 -- 172.21.15.17:0/2175476759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f0141310 con 0x7fd7f0145900 2024-03-16T10:10:51.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7e7fff640 1 --2- 172.21.15.17:0/2175476759 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7f013f1a0 0x7fd7f013e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:10:51.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7ef59e640 1 --2- 172.21.15.17:0/2175476759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f0145900 0x7fd7f01dd960 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd7e801af00 tx=0x7fd7e80195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:51.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7e77fe640 1 -- 172.21.15.17:0/2175476759 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80bd020 con 0x7fd7f0145900 2024-03-16T10:10:51.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7e77fe640 1 -- 172.21.15.17:0/2175476759 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7e80b3040 con 0x7fd7f0145900 2024-03-16T10:10:51.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7f5246640 1 -- 172.21.15.17:0/2175476759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f00b3220 con 0x7fd7f0145900 2024-03-16T10:10:51.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7e77fe640 1 -- 172.21.15.17:0/2175476759 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80b2070 con 0x7fd7f0145900 2024-03-16T10:10:51.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7f5246640 1 -- 172.21.15.17:0/2175476759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f01e1520 con 0x7fd7f0145900 2024-03-16T10:10:51.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.862+0000 7fd7f5246640 1 -- 172.21.15.17:0/2175476759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f006a130 con 0x7fd7f0145900 2024-03-16T10:10:51.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.866+0000 7fd7e77fe640 1 -- 172.21.15.17:0/2175476759 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd7e80c0020 con 0x7fd7f0145900 2024-03-16T10:10:51.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.866+0000 7fd7e77fe640 1 --2- 172.21.15.17:0/2175476759 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd7d0067660 0x7fd7d0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:51.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.866+0000 7fd7e77fe640 1 -- 172.21.15.17:0/2175476759 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd7e8120680 con 0x7fd7f0145900 2024-03-16T10:10:51.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.866+0000 7fd7eed9d640 1 --2- 172.21.15.17:0/2175476759 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd7d0067660 0x7fd7d0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:51.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.866+0000 7fd7eed9d640 1 --2- 172.21.15.17:0/2175476759 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd7d0067660 0x7fd7d0069b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0001810 tx=0x7fd7e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:51.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:51.870+0000 7fd7e77fe640 1 -- 172.21.15.17:0/2175476759 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd7f0141310 con 0x7fd7f0145900 2024-03-16T10:10:52.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:52.066+0000 7fd7f5246640 1 -- 172.21.15.17:0/2175476759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7f00b85e0 con 0x7fd7f0145900 2024-03-16T10:10:52.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:52.066+0000 7fd7e77fe640 1 -- 172.21.15.17:0/2175476759 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7fd7f01462c0 con 0x7fd7f0145900 2024-03-16T10:10:52.069 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:52.069 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3105182870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3105182870},{"type":"v1","addr":"172.21.15.17:6833","nonce":3105182870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-16T10:10:52.069 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 2 2024-03-16T10:10:52.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:52.070+0000 7fd7f5246640 1 -- 172.21.15.17:0/2175476759 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd7d0067660 msgr2=0x7fd7d0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:52.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:52.070+0000 7fd7f5246640 1 --2- 172.21.15.17:0/2175476759 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd7d0067660 0x7fd7d0069b20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0001810 tx=0x7fd7e0009040 comp rx=0 tx=0).stop 2024-03-16T10:10:52.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:52.070+0000 7fd7f5246640 1 -- 172.21.15.17:0/2175476759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f0145900 msgr2=0x7fd7f01dd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:52.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:52.070+0000 7fd7f5246640 1 --2- 172.21.15.17:0/2175476759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f0145900 0x7fd7f01dd960 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd7e801af00 tx=0x7fd7e80195f0 comp rx=0 tx=0).stop 2024-03-16T10:10:52.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:52.070+0000 7fd7f5246640 1 -- 172.21.15.17:0/2175476759 shutdown_connections 2024-03-16T10:10:52.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:52.070+0000 7fd7f5246640 1 --2- 172.21.15.17:0/2175476759 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd7d0067660 0x7fd7d0069b20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:52.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:52.070+0000 7fd7f5246640 1 --2- 172.21.15.17:0/2175476759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd7f0145900 0x7fd7f01dd960 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:52.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:52.070+0000 7fd7f5246640 1 --2- 172.21.15.17:0/2175476759 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd7f013f1a0 0x7fd7f013e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:52.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:52.070+0000 7fd7f5246640 1 --2- 172.21.15.17:0/2175476759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7f0072b80 0x7fd7f0140b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:52.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:52.070+0000 7fd7f5246640 1 -- 172.21.15.17:0/2175476759 >> 172.21.15.17:0/2175476759 conn(0x7fd7f00b7880 msgr2=0x7fd7f00b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:52.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:52.070+0000 7fd7f5246640 1 -- 172.21.15.17:0/2175476759 shutdown_connections 2024-03-16T10:10:52.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:52.070+0000 7fd7f5246640 1 -- 172.21.15.17:0/2175476759 wait complete. 2024-03-16T10:10:54.087 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:10:54.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7c512d640 1 Processor -- start 2024-03-16T10:10:54.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7c512d640 1 -- start start 2024-03-16T10:10:54.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7c512d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7c0072b80 0x7fb7c00b7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7c512d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb7c013f1a0 0x7fb7c013f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7c512d640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7c0070290 con 0x7fb7c0145900 2024-03-16T10:10:54.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7c512d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb7c0070570 con 0x7fb7c0072b80 2024-03-16T10:10:54.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7c512d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c0145440 con 0x7fb7c013f1a0 2024-03-16T10:10:54.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7bed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7c0072b80 0x7fb7c00b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7bed9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb7c0072b80 0x7fb7c00b7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43572/0 (socket says 172.21.15.17:43572) 2024-03-16T10:10:54.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7bed9d640 1 -- 172.21.15.17:0/1324805384 learned_addr learned my addr 172.21.15.17:0/1324805384 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:54.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7bf59e640 1 --2- 172.21.15.17:0/1324805384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb7c013f1a0 0x7fb7c013f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7bed9d640 1 -- 172.21.15.17:0/1324805384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb7c013f1a0 msgr2=0x7fb7c013f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:54.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7bed9d640 1 --2- 172.21.15.17:0/1324805384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb7c013f1a0 0x7fb7c013f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7bed9d640 1 -- 172.21.15.17:0/1324805384 >> v1:172.21.15.174:6789/0 conn(0x7fb7c0145900 legacy=0x7fb7c0145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7bed9d640 1 -- 172.21.15.17:0/1324805384 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c00b7670 con 0x7fb7c0072b80 2024-03-16T10:10:54.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.217+0000 7fb7bf59e640 1 --2- 172.21.15.17:0/1324805384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb7c013f1a0 0x7fb7c013f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:10:54.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.221+0000 7fb7bed9d640 1 --2- 172.21.15.17:0/1324805384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7c0072b80 0x7fb7c00b7130 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009870 tx=0x7fb7b001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c300a7ba34080247 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:54.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.221+0000 7fb7be59c640 1 -- 172.21.15.17:0/1324805384 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00b0070 con 0x7fb7c0072b80 2024-03-16T10:10:54.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.221+0000 7fb7be59c640 1 -- 172.21.15.17:0/1324805384 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b00ab600 con 0x7fb7c0072b80 2024-03-16T10:10:54.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.221+0000 7fb7be59c640 1 -- 172.21.15.17:0/1324805384 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00ab9d0 con 0x7fb7c0072b80 2024-03-16T10:10:54.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.221+0000 7fb7c512d640 1 -- 172.21.15.17:0/1324805384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7c0072b80 msgr2=0x7fb7c00b7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.221+0000 7fb7c512d640 1 --2- 172.21.15.17:0/1324805384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7c0072b80 0x7fb7c00b7130 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009870 tx=0x7fb7b001c440 comp rx=0 tx=0).stop 2024-03-16T10:10:54.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.221+0000 7fb7c512d640 1 -- 172.21.15.17:0/1324805384 shutdown_connections 2024-03-16T10:10:54.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.221+0000 7fb7c512d640 1 --2- 172.21.15.17:0/1324805384 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb7c013f1a0 0x7fb7c013f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.221+0000 7fb7c512d640 1 --2- 172.21.15.17:0/1324805384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7c0072b80 0x7fb7c00b7130 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.221+0000 7fb7c512d640 1 -- 172.21.15.17:0/1324805384 >> 172.21.15.17:0/1324805384 conn(0x7fb7c00b7880 msgr2=0x7fb7c00b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:54.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.221+0000 7fb7c512d640 1 -- 172.21.15.17:0/1324805384 shutdown_connections 2024-03-16T10:10:54.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.221+0000 7fb7c512d640 1 -- 172.21.15.17:0/1324805384 wait complete. 2024-03-16T10:10:54.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.221+0000 7fb7c512d640 1 Processor -- start 2024-03-16T10:10:54.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.221+0000 7fb7c512d640 1 -- start start 2024-03-16T10:10:54.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7c512d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb7c0072b80 0x7fb7c013b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7c512d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7c013f1a0 0x7fb7c0142450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7c512d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7c0145900 0x7fb7c01d75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7c512d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c0152500 con 0x7fb7c0145900 2024-03-16T10:10:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7c512d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c013fb00 con 0x7fb7c013f1a0 2024-03-16T10:10:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7c512d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c0152680 con 0x7fb7c0072b80 2024-03-16T10:10:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7bf59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7c0145900 0x7fb7c01d75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7bf59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7c0145900 0x7fb7c01d75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43582/0 (socket says 172.21.15.17:43582) 2024-03-16T10:10:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7bf59e640 1 -- 172.21.15.17:0/1050984490 learned_addr learned my addr 172.21.15.17:0/1050984490 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7bed9d640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb7c0072b80 0x7fb7c013b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7b7fff640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7c013f1a0 0x7fb7c0142450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7bf59e640 1 -- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb7c0072b80 msgr2=0x7fb7c013b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7bf59e640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb7c0072b80 0x7fb7c013b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7bf59e640 1 -- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7c013f1a0 msgr2=0x7fb7c0142450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7bf59e640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7c013f1a0 0x7fb7c0142450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7bf59e640 1 -- 172.21.15.17:0/1050984490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c0142e50 con 0x7fb7c0145900 2024-03-16T10:10:54.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7b7fff640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7c013f1a0 0x7fb7c0142450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:10:54.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7bf59e640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7c0145900 0x7fb7c01d75a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8007f20 tx=0x7fb7b800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:54.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7b77fe640 1 -- 172.21.15.17:0/1050984490 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b801a070 con 0x7fb7c0145900 2024-03-16T10:10:54.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7b77fe640 1 -- 172.21.15.17:0/1050984490 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b8003ba0 con 0x7fb7c0145900 2024-03-16T10:10:54.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7b77fe640 1 -- 172.21.15.17:0/1050984490 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8004030 con 0x7fb7c0145900 2024-03-16T10:10:54.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7c512d640 1 -- 172.21.15.17:0/1050984490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c00b4c20 con 0x7fb7c0145900 2024-03-16T10:10:54.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7c512d640 1 -- 172.21.15.17:0/1050984490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c013fd10 con 0x7fb7c0145900 2024-03-16T10:10:54.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7c512d640 1 -- 172.21.15.17:0/1050984490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c006a130 con 0x7fb7c0145900 2024-03-16T10:10:54.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7b77fe640 1 -- 172.21.15.17:0/1050984490 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fb7b801d070 con 0x7fb7c0145900 2024-03-16T10:10:54.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7b77fe640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb7a0067660 0x7fb7a0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.225+0000 7fb7b77fe640 1 -- 172.21.15.17:0/1050984490 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fb7b80857f0 con 0x7fb7c0145900 2024-03-16T10:10:54.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.229+0000 7fb7bed9d640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb7a0067660 0x7fb7a0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.229+0000 7fb7bed9d640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb7a0067660 0x7fb7a0069b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00ad000 tx=0x7fb7b00a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:54.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.233+0000 7fb7b77fe640 1 -- 172.21.15.17:0/1050984490 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb7b8010070 con 0x7fb7c0145900 2024-03-16T10:10:54.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.425+0000 7fb7c512d640 1 -- 172.21.15.17:0/1050984490 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7c0143550 con 0x7fb7c0145900 2024-03-16T10:10:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.425+0000 7fb7b77fe640 1 -- 172.21.15.17:0/1050984490 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2219 (secure 0 0 0) 0x7fb7b8056850 con 0x7fb7c0145900 2024-03-16T10:10:54.428 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:54.428 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1058712158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1058712158},{"type":"v1","addr":"172.21.15.174:6835","nonce":1058712158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3105182870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3105182870},{"type":"v1","addr":"172.21.15.17:6833","nonce":3105182870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-16T10:10:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 3 2024-03-16T10:10:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.425+0000 7fb7c512d640 1 -- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb7a0067660 msgr2=0x7fb7a0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.425+0000 7fb7c512d640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb7a0067660 0x7fb7a0069b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00ad000 tx=0x7fb7b00a6000 comp rx=0 tx=0).stop 2024-03-16T10:10:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.425+0000 7fb7c512d640 1 -- 172.21.15.17:0/1050984490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7c0145900 msgr2=0x7fb7c01d75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.425+0000 7fb7c512d640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7c0145900 0x7fb7c01d75a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8007f20 tx=0x7fb7b800d560 comp rx=0 tx=0).stop 2024-03-16T10:10:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.425+0000 7fb7c512d640 1 -- 172.21.15.17:0/1050984490 shutdown_connections 2024-03-16T10:10:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.425+0000 7fb7c512d640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb7a0067660 0x7fb7a0069b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.425+0000 7fb7c512d640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb7c0145900 0x7fb7c01d75a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.425+0000 7fb7c512d640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7c013f1a0 0x7fb7c0142450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.425+0000 7fb7c512d640 1 --2- 172.21.15.17:0/1050984490 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb7c0072b80 0x7fb7c013b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.425+0000 7fb7c512d640 1 -- 172.21.15.17:0/1050984490 >> 172.21.15.17:0/1050984490 conn(0x7fb7c00b7880 msgr2=0x7fb7c00b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.425+0000 7fb7c512d640 1 -- 172.21.15.17:0/1050984490 shutdown_connections 2024-03-16T10:10:54.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.425+0000 7fb7c512d640 1 -- 172.21.15.17:0/1050984490 wait complete. 2024-03-16T10:10:54.445 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:10:54.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.549+0000 7f943317d640 1 Processor -- start 2024-03-16T10:10:54.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f943317d640 1 -- start start 2024-03-16T10:10:54.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f943317d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f942c06d750 0x7f942c06f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f943317d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f942c0b2820 0x7f942c13eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f943317d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f942c06db30 con 0x7f942c145280 2024-03-16T10:10:54.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f943317d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f942c0b2d30 con 0x7f942c06d750 2024-03-16T10:10:54.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f943317d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f942c06fcf0 con 0x7f942c0b2820 2024-03-16T10:10:54.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f943171a640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f942c145280 0x7f942c13f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47502/0 (socket says 172.21.15.17:47502) 2024-03-16T10:10:54.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f943171a640 1 -- 172.21.15.17:0/3153764158 learned_addr learned my addr 172.21.15.17:0/3153764158 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:10:54.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f9430f19640 1 --2- 172.21.15.17:0/3153764158 >> v2:172.21.15.17:3300/0 conn(0x7f942c06d750 0x7f942c06f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f942bfff640 1 --2- 172.21.15.17:0/3153764158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f942c0b2820 0x7f942c13eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f942b7fe640 1 -- 172.21.15.17:0/3153764158 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3974272483 0 0) 0x7f942c06db30 con 0x7f942c145280 2024-03-16T10:10:54.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f942b7fe640 1 -- 172.21.15.17:0/3153764158 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9414003530 con 0x7f942c145280 2024-03-16T10:10:54.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f9430f19640 1 -- 172.21.15.17:0/3153764158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f942c0b2820 msgr2=0x7f942c13eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:54.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f9430f19640 1 --2- 172.21.15.17:0/3153764158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f942c0b2820 0x7f942c13eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f9430f19640 1 -- 172.21.15.17:0/3153764158 >> v1:172.21.15.17:6789/0 conn(0x7f942c145280 legacy=0x7f942c13f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.553+0000 7f9430f19640 1 -- 172.21.15.17:0/3153764158 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942c0afaf0 con 0x7f942c06d750 2024-03-16T10:10:54.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f9430f19640 1 --2- 172.21.15.17:0/3153764158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f942c06d750 0x7f942c06f7b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f941c0097f0 tx=0x7f941c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4c6463b1759421 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:54.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f942b7fe640 1 -- 172.21.15.17:0/3153764158 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c0b0070 con 0x7f942c06d750 2024-03-16T10:10:54.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f942b7fe640 1 -- 172.21.15.17:0/3153764158 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f941c0ab600 con 0x7f942c06d750 2024-03-16T10:10:54.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f942b7fe640 1 -- 172.21.15.17:0/3153764158 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f941c0ab9d0 con 0x7f942c06d750 2024-03-16T10:10:54.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 -- 172.21.15.17:0/3153764158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f942c06d750 msgr2=0x7f942c06f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 --2- 172.21.15.17:0/3153764158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f942c06d750 0x7f942c06f7b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f941c0097f0 tx=0x7f941c01c380 comp rx=0 tx=0).stop 2024-03-16T10:10:54.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 -- 172.21.15.17:0/3153764158 shutdown_connections 2024-03-16T10:10:54.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 --2- 172.21.15.17:0/3153764158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f942c0b2820 0x7f942c13eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 --2- 172.21.15.17:0/3153764158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f942c06d750 0x7f942c06f7b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 -- 172.21.15.17:0/3153764158 >> 172.21.15.17:0/3153764158 conn(0x7f942c0b55c0 msgr2=0x7f942c0b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:54.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 -- 172.21.15.17:0/3153764158 shutdown_connections 2024-03-16T10:10:54.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 -- 172.21.15.17:0/3153764158 wait complete. 2024-03-16T10:10:54.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 Processor -- start 2024-03-16T10:10:54.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 -- start start 2024-03-16T10:10:54.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f942c0b2820 0x7f942c147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f942c145280 0x7f942c1df860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f942c142540 0x7f942c14eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f942c152550 con 0x7f942c145280 2024-03-16T10:10:54.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f942c0b49f0 con 0x7f942c0b2820 2024-03-16T10:10:54.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f942c1526d0 con 0x7f942c142540 2024-03-16T10:10:54.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f942bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f942c145280 0x7f942c1df860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f942bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f942c145280 0x7f942c1df860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43602/0 (socket says 172.21.15.17:43602) 2024-03-16T10:10:54.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f942bfff640 1 -- 172.21.15.17:0/1477243480 learned_addr learned my addr 172.21.15.17:0/1477243480 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943171a640 1 --2- 172.21.15.17:0/1477243480 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f942c142540 0x7f942c14eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f9430f19640 1 --2- 172.21.15.17:0/1477243480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f942c0b2820 0x7f942c147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f942bfff640 1 -- 172.21.15.17:0/1477243480 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f942c142540 msgr2=0x7f942c14eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f942bfff640 1 --2- 172.21.15.17:0/1477243480 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f942c142540 0x7f942c14eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f942bfff640 1 -- 172.21.15.17:0/1477243480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f942c0b2820 msgr2=0x7f942c147400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f942bfff640 1 --2- 172.21.15.17:0/1477243480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f942c0b2820 0x7f942c147400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f942bfff640 1 -- 172.21.15.17:0/1477243480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f942c148390 con 0x7f942c145280 2024-03-16T10:10:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f942bfff640 1 --2- 172.21.15.17:0/1477243480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f942c145280 0x7f942c1df860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f942000d7d0 tx=0x7f942000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f94297fa640 1 -- 172.21.15.17:0/1477243480 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942000b6c0 con 0x7f942c145280 2024-03-16T10:10:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f94297fa640 1 -- 172.21.15.17:0/1477243480 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9420019070 con 0x7f942c145280 2024-03-16T10:10:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 -- 172.21.15.17:0/1477243480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f942c146710 con 0x7f942c145280 2024-03-16T10:10:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f94297fa640 1 -- 172.21.15.17:0/1477243480 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942000baa0 con 0x7f942c145280 2024-03-16T10:10:54.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.557+0000 7f943317d640 1 -- 172.21.15.17:0/1477243480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f942c1463e0 con 0x7f942c145280 2024-03-16T10:10:54.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.561+0000 7f943317d640 1 -- 172.21.15.17:0/1477243480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f942c068360 con 0x7f942c145280 2024-03-16T10:10:54.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.561+0000 7f94297fa640 1 -- 172.21.15.17:0/1477243480 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f942000bc40 con 0x7f942c145280 2024-03-16T10:10:54.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.561+0000 7f94297fa640 1 --2- 172.21.15.17:0/1477243480 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9418067660 0x7f9418069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.561+0000 7f94297fa640 1 -- 172.21.15.17:0/1477243480 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f9420085970 con 0x7f942c145280 2024-03-16T10:10:54.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.561+0000 7f9430f19640 1 --2- 172.21.15.17:0/1477243480 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9418067660 0x7f9418069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.561+0000 7f9430f19640 1 --2- 172.21.15.17:0/1477243480 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9418067660 0x7f9418069b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f941c0ad000 tx=0x7f941c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:54.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.565+0000 7f94297fa640 1 -- 172.21.15.17:0/1477243480 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f94200569d0 con 0x7f942c145280 2024-03-16T10:10:54.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.721+0000 7f943317d640 1 -- 172.21.15.17:0/1477243480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f942c146d40 con 0x7f942c145280 2024-03-16T10:10:54.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.721+0000 7f94297fa640 1 -- 172.21.15.17:0/1477243480 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12884 (secure 0 0 0) 0x7f94200562d0 con 0x7f942c145280 2024-03-16T10:10:54.724 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:54.724 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:48.527781+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:10:54.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.721+0000 7f943317d640 1 -- 172.21.15.17:0/1477243480 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9418067660 msgr2=0x7f9418069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.721+0000 7f943317d640 1 --2- 172.21.15.17:0/1477243480 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9418067660 0x7f9418069b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f941c0ad000 tx=0x7f941c008040 comp rx=0 tx=0).stop 2024-03-16T10:10:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.721+0000 7f943317d640 1 -- 172.21.15.17:0/1477243480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f942c145280 msgr2=0x7f942c1df860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.721+0000 7f943317d640 1 --2- 172.21.15.17:0/1477243480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f942c145280 0x7f942c1df860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f942000d7d0 tx=0x7f942000dc90 comp rx=0 tx=0).stop 2024-03-16T10:10:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.721+0000 7f943317d640 1 -- 172.21.15.17:0/1477243480 shutdown_connections 2024-03-16T10:10:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.721+0000 7f943317d640 1 --2- 172.21.15.17:0/1477243480 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9418067660 0x7f9418069b20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.721+0000 7f943317d640 1 --2- 172.21.15.17:0/1477243480 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f942c142540 0x7f942c14eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.721+0000 7f943317d640 1 --2- 172.21.15.17:0/1477243480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f942c145280 0x7f942c1df860 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.721+0000 7f943317d640 1 --2- 172.21.15.17:0/1477243480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f942c0b2820 0x7f942c147400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.721+0000 7f943317d640 1 -- 172.21.15.17:0/1477243480 >> 172.21.15.17:0/1477243480 conn(0x7f942c0b55c0 msgr2=0x7f942c143f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.721+0000 7f943317d640 1 -- 172.21.15.17:0/1477243480 shutdown_connections 2024-03-16T10:10:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.721+0000 7f943317d640 1 -- 172.21.15.17:0/1477243480 wait complete. 2024-03-16T10:10:54.741 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:10:54.742 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:10:54.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.861+0000 7fd302db5640 1 Processor -- start 2024-03-16T10:10:54.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd302db5640 1 -- start start 2024-03-16T10:10:54.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd302db5640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd2fc0b59a0 0x7fd2fc150b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd302db5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2fc06f2b0 con 0x7fd2fc06eed0 2024-03-16T10:10:54.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd302db5640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2fc06d600 con 0x7fd2fc06e000 2024-03-16T10:10:54.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd302db5640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fd2fc06dc50 con 0x7fd2fc0b59a0 2024-03-16T10:10:54.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd300b51640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd2fc06eed0 0x7fd2fc072b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47510/0 (socket says 172.21.15.17:47510) 2024-03-16T10:10:54.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd300b51640 1 -- 172.21.15.17:0/1591069933 learned_addr learned my addr 172.21.15.17:0/1591069933 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:10:54.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd301352640 1 --2- 172.21.15.17:0/1591069933 >> v2:172.21.15.174:3300/0 conn(0x7fd2fc0b59a0 0x7fd2fc150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 -- 172.21.15.17:0/1591069933 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3149502219 0 0) 0x7fd2fc06f2b0 con 0x7fd2fc06eed0 2024-03-16T10:10:54.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 -- 172.21.15.17:0/1591069933 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2e4003530 con 0x7fd2fc06eed0 2024-03-16T10:10:54.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 -- 172.21.15.17:0/1591069933 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2629057097 0 0) 0x7fd2e4003530 con 0x7fd2fc06eed0 2024-03-16T10:10:54.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 -- 172.21.15.17:0/1591069933 >> v2:172.21.15.174:3300/0 conn(0x7fd2fc0b59a0 msgr2=0x7fd2fc150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:54.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 --2- 172.21.15.17:0/1591069933 >> v2:172.21.15.174:3300/0 conn(0x7fd2fc0b59a0 0x7fd2fc150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 -- 172.21.15.17:0/1591069933 >> v1:172.21.15.174:6789/0 conn(0x7fd2fc06e000 legacy=0x7fd2fc13f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 -- 172.21.15.17:0/1591069933 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc06d8e0 con 0x7fd2fc06eed0 2024-03-16T10:10:54.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 -- 172.21.15.17:0/1591069933 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fd2f4001a00 con 0x7fd2fc06eed0 2024-03-16T10:10:54.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 -- 172.21.15.17:0/1591069933 >> v1:172.21.15.17:6789/0 conn(0x7fd2fc06eed0 legacy=0x7fd2fc072b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 --2- 172.21.15.17:0/1591069933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2e40039c0 0x7fd2e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 --2- 172.21.15.17:0/1591069933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2e4004810 0x7fd2e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 --2- 172.21.15.17:0/1591069933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2e40051d0 0x7fd2e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 -- 172.21.15.17:0/1591069933 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc06dc50 con 0x7fd2e40039c0 2024-03-16T10:10:54.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 -- 172.21.15.17:0/1591069933 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e400c080 con 0x7fd2e4004810 2024-03-16T10:10:54.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fb7fe640 1 -- 172.21.15.17:0/1591069933 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd2e400c280 con 0x7fd2e40051d0 2024-03-16T10:10:54.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd300b51640 1 --2- 172.21.15.17:0/1591069933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2e40039c0 0x7fd2e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd2fbfff640 1 --2- 172.21.15.17:0/1591069933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2e4004810 0x7fd2e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd301352640 1 --2- 172.21.15.17:0/1591069933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2e40051d0 0x7fd2e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd300b51640 1 -- 172.21.15.17:0/1591069933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2e40051d0 msgr2=0x7fd2e400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:54.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd300b51640 1 --2- 172.21.15.17:0/1591069933 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2e40051d0 0x7fd2e400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd300b51640 1 -- 172.21.15.17:0/1591069933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2e4004810 msgr2=0x7fd2e4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:54.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd300b51640 1 --2- 172.21.15.17:0/1591069933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2e4004810 0x7fd2e4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd300b51640 1 -- 172.21.15.17:0/1591069933 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e4003530 con 0x7fd2e40039c0 2024-03-16T10:10:54.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.865+0000 7fd301352640 1 -- 172.21.15.17:0/1591069933 reap_dead start 2024-03-16T10:10:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd300b51640 1 --2- 172.21.15.17:0/1591069933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2e40039c0 0x7fd2e4003de0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400a840 tx=0x7fd2f401fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c39a1b7e66baedb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2fb7fe640 1 -- 172.21.15.17:0/1591069933 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f400a460 con 0x7fd2e40039c0 2024-03-16T10:10:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2fb7fe640 1 -- 172.21.15.17:0/1591069933 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2f40aa670 con 0x7fd2e40039c0 2024-03-16T10:10:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2fb7fe640 1 -- 172.21.15.17:0/1591069933 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f40aaaa0 con 0x7fd2e40039c0 2024-03-16T10:10:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 -- 172.21.15.17:0/1591069933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2e40039c0 msgr2=0x7fd2e4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 --2- 172.21.15.17:0/1591069933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2e40039c0 0x7fd2e4003de0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400a840 tx=0x7fd2f401fa40 comp rx=0 tx=0).stop 2024-03-16T10:10:54.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 -- 172.21.15.17:0/1591069933 shutdown_connections 2024-03-16T10:10:54.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 --2- 172.21.15.17:0/1591069933 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2e40039c0 0x7fd2e4003de0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 -- 172.21.15.17:0/1591069933 >> 172.21.15.17:0/1591069933 conn(0x7fd2fc0b65d0 msgr2=0x7fd2fc0b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:54.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 -- 172.21.15.17:0/1591069933 shutdown_connections 2024-03-16T10:10:54.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 -- 172.21.15.17:0/1591069933 wait complete. 2024-03-16T10:10:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 Processor -- start 2024-03-16T10:10:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 -- start start 2024-03-16T10:10:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2e40039c0 0x7fd2fc0b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2fc09f0b0 0x7fd2fc1e0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2fc137010 0x7fd2fc137490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc06e580 con 0x7fd2fc09f0b0 2024-03-16T10:10:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2fc06e400 con 0x7fd2fc137010 2024-03-16T10:10:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd2fc06e700 con 0x7fd2e40039c0 2024-03-16T10:10:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2fc09f0b0 0x7fd2fc1e0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2fbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2fc09f0b0 0x7fd2fc1e0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43634/0 (socket says 172.21.15.17:43634) 2024-03-16T10:10:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2fbfff640 1 -- 172.21.15.17:0/1724948438 learned_addr learned my addr 172.21.15.17:0/1724948438 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd301352640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2fc137010 0x7fd2fc137490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd300b51640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2e40039c0 0x7fd2fc0b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2fbfff640 1 -- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2e40039c0 msgr2=0x7fd2fc0b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2fbfff640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2e40039c0 0x7fd2fc0b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2fbfff640 1 -- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2fc137010 msgr2=0x7fd2fc137490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2fbfff640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2fc137010 0x7fd2fc137490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2fbfff640 1 -- 172.21.15.17:0/1724948438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2fc0b06d0 con 0x7fd2fc09f0b0 2024-03-16T10:10:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd300b51640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2e40039c0 0x7fd2fc0b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:10:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2fbfff640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2fc09f0b0 0x7fd2fc1e0160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000e440 tx=0x7fd2f000e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2f97fa640 1 -- 172.21.15.17:0/1724948438 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0019070 con 0x7fd2fc09f0b0 2024-03-16T10:10:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2f97fa640 1 -- 172.21.15.17:0/1724948438 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2f00052e0 con 0x7fd2fc09f0b0 2024-03-16T10:10:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd2f97fa640 1 -- 172.21.15.17:0/1724948438 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0003840 con 0x7fd2fc09f0b0 2024-03-16T10:10:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 -- 172.21.15.17:0/1724948438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2fc1e06a0 con 0x7fd2fc09f0b0 2024-03-16T10:10:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.869+0000 7fd302db5640 1 -- 172.21.15.17:0/1724948438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2fc0aec20 con 0x7fd2fc09f0b0 2024-03-16T10:10:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.873+0000 7fd302db5640 1 -- 172.21.15.17:0/1724948438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2fc1379d0 con 0x7fd2fc09f0b0 2024-03-16T10:10:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.873+0000 7fd2f97fa640 1 -- 172.21.15.17:0/1724948438 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd2f001c070 con 0x7fd2fc09f0b0 2024-03-16T10:10:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.873+0000 7fd2f97fa640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd2e8067660 0x7fd2e8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.873+0000 7fd2f97fa640 1 -- 172.21.15.17:0/1724948438 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd2f00849a0 con 0x7fd2fc09f0b0 2024-03-16T10:10:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.873+0000 7fd300b51640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd2e8067660 0x7fd2e8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.873+0000 7fd300b51640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd2e8067660 0x7fd2e8069b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40b0000 tx=0x7fd2f4007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:54.877+0000 7fd2f97fa640 1 -- 172.21.15.17:0/1724948438 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd2fc1379d0 con 0x7fd2fc09f0b0 2024-03-16T10:10:55.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.033+0000 7fd302db5640 1 -- 172.21.15.17:0/1724948438 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd2fc0b8410 con 0x7fd2fc09f0b0 2024-03-16T10:10:55.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.033+0000 7fd2f97fa640 1 -- 172.21.15.17:0/1724948438 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7fd2f00553b0 con 0x7fd2fc09f0b0 2024-03-16T10:10:55.038 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:10:55.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.037+0000 7fd302db5640 1 -- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd2e8067660 msgr2=0x7fd2e8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:55.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.037+0000 7fd302db5640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd2e8067660 0x7fd2e8069b20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40b0000 tx=0x7fd2f4007040 comp rx=0 tx=0).stop 2024-03-16T10:10:55.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.037+0000 7fd302db5640 1 -- 172.21.15.17:0/1724948438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2fc09f0b0 msgr2=0x7fd2fc1e0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:55.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.037+0000 7fd302db5640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2fc09f0b0 0x7fd2fc1e0160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000e440 tx=0x7fd2f000e900 comp rx=0 tx=0).stop 2024-03-16T10:10:55.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.037+0000 7fd302db5640 1 -- 172.21.15.17:0/1724948438 shutdown_connections 2024-03-16T10:10:55.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.037+0000 7fd302db5640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd2e8067660 0x7fd2e8069b20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.037+0000 7fd302db5640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2fc137010 0x7fd2fc137490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.037+0000 7fd302db5640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2fc09f0b0 0x7fd2fc1e0160 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.037+0000 7fd302db5640 1 --2- 172.21.15.17:0/1724948438 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd2e40039c0 0x7fd2fc0b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.037+0000 7fd302db5640 1 -- 172.21.15.17:0/1724948438 >> 172.21.15.17:0/1724948438 conn(0x7fd2fc0b65d0 msgr2=0x7fd2fc1452b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:55.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.037+0000 7fd302db5640 1 -- 172.21.15.17:0/1724948438 shutdown_connections 2024-03-16T10:10:55.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.037+0000 7fd302db5640 1 -- 172.21.15.17:0/1724948438 wait complete. 2024-03-16T10:10:55.056 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-16T10:10:55.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.173+0000 7f79a9bab640 1 Processor -- start 2024-03-16T10:10:55.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.173+0000 7f79a9bab640 1 -- start start 2024-03-16T10:10:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.173+0000 7f79a9bab640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f79a4145900 0x7f79a4145d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.173+0000 7f79a9bab640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a4070290 con 0x7f79a4072b80 2024-03-16T10:10:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.173+0000 7f79a9bab640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79a4070570 con 0x7f79a413f1a0 2024-03-16T10:10:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.173+0000 7f79a9bab640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f79a4145440 con 0x7f79a4145900 2024-03-16T10:10:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.173+0000 7f79a37fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f79a4072b80 0x7f79a40b7130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47526/0 (socket says 172.21.15.17:47526) 2024-03-16T10:10:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.173+0000 7f79a37fe640 1 -- 172.21.15.17:0/664414791 learned_addr learned my addr 172.21.15.17:0/664414791 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:10:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.173+0000 7f79a2ffd640 1 --2- 172.21.15.17:0/664414791 >> v2:172.21.15.174:3300/0 conn(0x7f79a4145900 0x7f79a4145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.173+0000 7f79a27fc640 1 -- 172.21.15.17:0/664414791 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1047106525 0 0) 0x7f79a4070290 con 0x7f79a4072b80 2024-03-16T10:10:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.173+0000 7f79a27fc640 1 -- 172.21.15.17:0/664414791 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f798c003530 con 0x7f79a4072b80 2024-03-16T10:10:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 -- 172.21.15.17:0/664414791 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3415053825 0 0) 0x7f798c003530 con 0x7f79a4072b80 2024-03-16T10:10:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 -- 172.21.15.17:0/664414791 >> v2:172.21.15.174:3300/0 conn(0x7f79a4145900 msgr2=0x7f79a4145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 --2- 172.21.15.17:0/664414791 >> v2:172.21.15.174:3300/0 conn(0x7f79a4145900 0x7f79a4145d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 -- 172.21.15.17:0/664414791 >> v1:172.21.15.174:6789/0 conn(0x7f79a413f1a0 legacy=0x7f79a413f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 -- 172.21.15.17:0/664414791 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a41462c0 con 0x7f79a4072b80 2024-03-16T10:10:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 -- 172.21.15.17:0/664414791 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f799c001a00 con 0x7f79a4072b80 2024-03-16T10:10:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 -- 172.21.15.17:0/664414791 >> v1:172.21.15.17:6789/0 conn(0x7f79a4072b80 legacy=0x7f79a40b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 --2- 172.21.15.17:0/664414791 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f798c0039c0 0x7f798c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 --2- 172.21.15.17:0/664414791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c004810 0x7f798c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 --2- 172.21.15.17:0/664414791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f798c0052f0 0x7f798c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 -- 172.21.15.17:0/664414791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f79a4145440 con 0x7f798c004810 2024-03-16T10:10:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 -- 172.21.15.17:0/664414791 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f798c00c1c0 con 0x7f798c0052f0 2024-03-16T10:10:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 -- 172.21.15.17:0/664414791 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f798c00c3c0 con 0x7f798c0039c0 2024-03-16T10:10:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a2ffd640 1 --2- 172.21.15.17:0/664414791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c004810 0x7f798c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a37fe640 1 --2- 172.21.15.17:0/664414791 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f798c0039c0 0x7f798c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a3fff640 1 --2- 172.21.15.17:0/664414791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f798c0052f0 0x7f798c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a2ffd640 1 -- 172.21.15.17:0/664414791 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f798c0039c0 msgr2=0x7f798c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:55.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a2ffd640 1 --2- 172.21.15.17:0/664414791 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f798c0039c0 0x7f798c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a2ffd640 1 -- 172.21.15.17:0/664414791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f798c0052f0 msgr2=0x7f798c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:55.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a2ffd640 1 --2- 172.21.15.17:0/664414791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f798c0052f0 0x7f798c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a2ffd640 1 -- 172.21.15.17:0/664414791 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c003530 con 0x7f798c004810 2024-03-16T10:10:55.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a3fff640 1 --2- 172.21.15.17:0/664414791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f798c0052f0 0x7f798c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:10:55.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a3fff640 1 -- 172.21.15.17:0/664414791 reap_dead start 2024-03-16T10:10:55.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a2ffd640 1 --2- 172.21.15.17:0/664414791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c004810 0x7f798c004c90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7998009ab0 tx=0x7f799801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4db124ae1b377fa server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:55.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 -- 172.21.15.17:0/664414791 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79980096b0 con 0x7f798c004810 2024-03-16T10:10:55.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 -- 172.21.15.17:0/664414791 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79980a6430 con 0x7f798c004810 2024-03-16T10:10:55.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.177+0000 7f79a27fc640 1 -- 172.21.15.17:0/664414791 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79980a6820 con 0x7f798c004810 2024-03-16T10:10:55.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 -- 172.21.15.17:0/664414791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c004810 msgr2=0x7f798c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:55.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 --2- 172.21.15.17:0/664414791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c004810 0x7f798c004c90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7998009ab0 tx=0x7f799801ca50 comp rx=0 tx=0).stop 2024-03-16T10:10:55.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 -- 172.21.15.17:0/664414791 shutdown_connections 2024-03-16T10:10:55.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 --2- 172.21.15.17:0/664414791 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f798c004810 0x7f798c004c90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 -- 172.21.15.17:0/664414791 >> 172.21.15.17:0/664414791 conn(0x7f79a40b7880 msgr2=0x7f79a40b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:55.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 -- 172.21.15.17:0/664414791 shutdown_connections 2024-03-16T10:10:55.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 -- 172.21.15.17:0/664414791 wait complete. 2024-03-16T10:10:55.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 Processor -- start 2024-03-16T10:10:55.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 -- start start 2024-03-16T10:10:55.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f798c004810 0x7f79a41d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79a41400e0 0x7f79a4140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79a413c300 0x7f79a413c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f79a40b76c0 con 0x7f79a413c300 2024-03-16T10:10:55.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79a40b7540 con 0x7f798c004810 2024-03-16T10:10:55.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f79a41491c0 con 0x7f79a41400e0 2024-03-16T10:10:55.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79a413c300 0x7f79a413c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79a413c300 0x7f79a413c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43662/0 (socket says 172.21.15.17:43662) 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a3fff640 1 -- 172.21.15.17:0/1435909451 learned_addr learned my addr 172.21.15.17:0/1435909451 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a37fe640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f798c004810 0x7f79a41d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a2ffd640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79a41400e0 0x7f79a4140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a3fff640 1 -- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79a41400e0 msgr2=0x7f79a4140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a3fff640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79a41400e0 0x7f79a4140560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a3fff640 1 -- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f798c004810 msgr2=0x7f79a41d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a3fff640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f798c004810 0x7f79a41d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a3fff640 1 -- 172.21.15.17:0/1435909451 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a4142200 con 0x7f79a413c300 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a2ffd640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79a41400e0 0x7f79a4140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a3fff640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79a413c300 0x7f79a413c780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f79a406fbf0 tx=0x7f799400d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f798bfff640 1 -- 172.21.15.17:0/1435909451 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994018070 con 0x7f79a413c300 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f798bfff640 1 -- 172.21.15.17:0/1435909451 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f799400de80 con 0x7f79a413c300 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f798bfff640 1 -- 172.21.15.17:0/1435909451 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7994003840 con 0x7f79a413c300 2024-03-16T10:10:55.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 -- 172.21.15.17:0/1435909451 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a41d2530 con 0x7f79a413c300 2024-03-16T10:10:55.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.181+0000 7f79a9bab640 1 -- 172.21.15.17:0/1435909451 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a41480b0 con 0x7f79a413c300 2024-03-16T10:10:55.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.185+0000 7f79a9bab640 1 -- 172.21.15.17:0/1435909451 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79a406a130 con 0x7f79a413c300 2024-03-16T10:10:55.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.185+0000 7f798bfff640 1 -- 172.21.15.17:0/1435909451 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f799401b070 con 0x7f79a413c300 2024-03-16T10:10:55.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.185+0000 7f798bfff640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7990067690 0x7f7990069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.185+0000 7f798bfff640 1 -- 172.21.15.17:0/1435909451 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f7994084af0 con 0x7f79a413c300 2024-03-16T10:10:55.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.185+0000 7f79a37fe640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7990067690 0x7f7990069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.185+0000 7f79a37fe640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7990067690 0x7f7990069b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79a413d230 tx=0x7f799c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:55.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.189+0000 7f798bfff640 1 -- 172.21.15.17:0/1435909451 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f79a4070570 con 0x7f79a413c300 2024-03-16T10:10:55.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.353+0000 7f79a9bab640 1 -- 172.21.15.17:0/1435909451 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f79a40b85e0 con 0x7f79a413c300 2024-03-16T10:10:55.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.353+0000 7f798bfff640 1 -- 172.21.15.17:0/1435909451 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f7994055ab0 con 0x7f79a413c300 2024-03-16T10:10:55.357 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:55.358 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-16T10:10:55.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:10:55.358 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-16T10:10:55.358 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-16T10:10:55.358 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:10:55.358 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-16T10:10:55.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.357+0000 7f79a9bab640 1 -- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7990067690 msgr2=0x7f7990069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:55.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.357+0000 7f79a9bab640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7990067690 0x7f7990069b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f79a413d230 tx=0x7f799c009040 comp rx=0 tx=0).stop 2024-03-16T10:10:55.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.357+0000 7f79a9bab640 1 -- 172.21.15.17:0/1435909451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79a413c300 msgr2=0x7f79a413c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:55.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.357+0000 7f79a9bab640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79a413c300 0x7f79a413c780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f79a406fbf0 tx=0x7f799400d680 comp rx=0 tx=0).stop 2024-03-16T10:10:55.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.357+0000 7f79a9bab640 1 -- 172.21.15.17:0/1435909451 shutdown_connections 2024-03-16T10:10:55.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.357+0000 7f79a9bab640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7990067690 0x7f7990069b50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.357+0000 7f79a9bab640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79a413c300 0x7f79a413c780 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.357+0000 7f79a9bab640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79a41400e0 0x7f79a4140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.357+0000 7f79a9bab640 1 --2- 172.21.15.17:0/1435909451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f798c004810 0x7f79a41d1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.357+0000 7f79a9bab640 1 -- 172.21.15.17:0/1435909451 >> 172.21.15.17:0/1435909451 conn(0x7f79a40b7880 msgr2=0x7f79a40b7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:55.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.357+0000 7f79a9bab640 1 -- 172.21.15.17:0/1435909451 shutdown_connections 2024-03-16T10:10:55.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.357+0000 7f79a9bab640 1 -- 172.21.15.17:0/1435909451 wait complete. 2024-03-16T10:10:55.377 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-16T10:10:55.377 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-16T10:10:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.501+0000 7f1d38218640 1 Processor -- start 2024-03-16T10:10:55.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.501+0000 7f1d38218640 1 -- start start 2024-03-16T10:10:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d38218640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d300b4650 0x7f1d3014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d38218640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1d300b7650 0x7f1d301412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d38218640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d30141c50 con 0x7f1d30141820 2024-03-16T10:10:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d38218640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f1d300b4a30 con 0x7f1d300b7650 2024-03-16T10:10:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d38218640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3014abe0 con 0x7f1d300b4650 2024-03-16T10:10:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d367b5640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1d30141820 0x7f1d301474d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47528/0 (socket says 172.21.15.17:47528) 2024-03-16T10:10:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d367b5640 1 -- 172.21.15.17:0/206013252 learned_addr learned my addr 172.21.15.17:0/206013252 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:10:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d357b3640 1 --2- 172.21.15.17:0/206013252 >> v2:172.21.15.174:3300/0 conn(0x7f1d300b7650 0x7f1d301412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d35fb4640 1 --2- 172.21.15.17:0/206013252 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d300b4650 0x7f1d3014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 -- 172.21.15.17:0/206013252 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2730860150 0 0) 0x7f1d30141c50 con 0x7f1d30141820 2024-03-16T10:10:55.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 -- 172.21.15.17:0/206013252 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d1c003530 con 0x7f1d30141820 2024-03-16T10:10:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 -- 172.21.15.17:0/206013252 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3522626666 0 0) 0x7f1d1c003530 con 0x7f1d30141820 2024-03-16T10:10:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 -- 172.21.15.17:0/206013252 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d300b4650 msgr2=0x7f1d3014b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 --2- 172.21.15.17:0/206013252 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d300b4650 0x7f1d3014b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 -- 172.21.15.17:0/206013252 >> v2:172.21.15.174:3300/0 conn(0x7f1d300b7650 msgr2=0x7f1d301412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 --2- 172.21.15.17:0/206013252 >> v2:172.21.15.174:3300/0 conn(0x7f1d300b7650 0x7f1d301412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 -- 172.21.15.17:0/206013252 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3014aec0 con 0x7f1d30141820 2024-03-16T10:10:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 -- 172.21.15.17:0/206013252 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f1d28001a00 con 0x7f1d30141820 2024-03-16T10:10:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 -- 172.21.15.17:0/206013252 >> v1:172.21.15.17:6789/0 conn(0x7f1d30141820 legacy=0x7f1d301474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 --2- 172.21.15.17:0/206013252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d1c0039c0 0x7f1d1c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 --2- 172.21.15.17:0/206013252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1c004810 0x7f1d1c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 --2- 172.21.15.17:0/206013252 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d300b4650 0x7f1d1c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 -- 172.21.15.17:0/206013252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1d300b4a30 con 0x7f1d1c0039c0 2024-03-16T10:10:55.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 -- 172.21.15.17:0/206013252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3014abe0 con 0x7f1d1c004810 2024-03-16T10:10:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 -- 172.21.15.17:0/206013252 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1d1c00ba20 con 0x7f1d300b4650 2024-03-16T10:10:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d35fb4640 1 --2- 172.21.15.17:0/206013252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d1c0039c0 0x7f1d1c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d367b5640 1 --2- 172.21.15.17:0/206013252 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d300b4650 0x7f1d1c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d357b3640 1 --2- 172.21.15.17:0/206013252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1c004810 0x7f1d1c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d35fb4640 1 -- 172.21.15.17:0/206013252 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d300b4650 msgr2=0x7f1d1c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d35fb4640 1 --2- 172.21.15.17:0/206013252 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d300b4650 0x7f1d1c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d35fb4640 1 -- 172.21.15.17:0/206013252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1c004810 msgr2=0x7f1d1c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d35fb4640 1 --2- 172.21.15.17:0/206013252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1c004810 0x7f1d1c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d35fb4640 1 -- 172.21.15.17:0/206013252 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d1c003530 con 0x7f1d1c0039c0 2024-03-16T10:10:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d367b5640 1 --2- 172.21.15.17:0/206013252 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d300b4650 0x7f1d1c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:10:55.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d357b3640 1 --2- 172.21.15.17:0/206013252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1c004810 0x7f1d1c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:10:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d35fb4640 1 --2- 172.21.15.17:0/206013252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d1c0039c0 0x7f1d1c003de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009b10 tx=0x7f1d2401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8158b789499278d2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 -- 172.21.15.17:0/206013252 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d240096b0 con 0x7f1d1c0039c0 2024-03-16T10:10:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 -- 172.21.15.17:0/206013252 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d240a9470 con 0x7f1d1c0039c0 2024-03-16T10:10:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.505+0000 7f1d34fb2640 1 -- 172.21.15.17:0/206013252 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d240a9860 con 0x7f1d1c0039c0 2024-03-16T10:10:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 -- 172.21.15.17:0/206013252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d1c0039c0 msgr2=0x7f1d1c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 --2- 172.21.15.17:0/206013252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d1c0039c0 0x7f1d1c003de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009b10 tx=0x7f1d2401ca70 comp rx=0 tx=0).stop 2024-03-16T10:10:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d367b5640 1 -- 172.21.15.17:0/206013252 reap_dead start 2024-03-16T10:10:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 -- 172.21.15.17:0/206013252 shutdown_connections 2024-03-16T10:10:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 -- 172.21.15.17:0/206013252 >> 172.21.15.17:0/206013252 conn(0x7f1d300bccd0 msgr2=0x7f1d300bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 -- 172.21.15.17:0/206013252 shutdown_connections 2024-03-16T10:10:55.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 -- 172.21.15.17:0/206013252 wait complete. 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 Processor -- start 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 -- start start 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1c0039c0 0x7f1d301d4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d301d52b0 0x7f1d301e36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d301e3c10 0x7f1d3013cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1d300b5b70 con 0x7f1d301d52b0 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1d300b59f0 con 0x7f1d1c0039c0 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3014b7f0 con 0x7f1d301e3c10 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d357b3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d301d52b0 0x7f1d301e36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d357b3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d301d52b0 0x7f1d301e36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43672/0 (socket says 172.21.15.17:43672) 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d357b3640 1 -- 172.21.15.17:0/306437188 learned_addr learned my addr 172.21.15.17:0/306437188 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d35fb4640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1c0039c0 0x7f1d301d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d367b5640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d301e3c10 0x7f1d3013cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d357b3640 1 -- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d301e3c10 msgr2=0x7f1d3013cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:55.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d357b3640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d301e3c10 0x7f1d3013cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d357b3640 1 -- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1c0039c0 msgr2=0x7f1d301d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:55.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d357b3640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1c0039c0 0x7f1d301d4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:55.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d357b3640 1 -- 172.21.15.17:0/306437188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d301d5800 con 0x7f1d301d52b0 2024-03-16T10:10:55.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d367b5640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d301e3c10 0x7f1d3013cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:10:55.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d357b3640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d301d52b0 0x7f1d301e36d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c010950 tx=0x7f1d2c010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:55.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d227fc640 1 -- 172.21.15.17:0/306437188 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c00f040 con 0x7f1d301d52b0 2024-03-16T10:10:55.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d227fc640 1 -- 172.21.15.17:0/306437188 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d2c00ec40 con 0x7f1d301d52b0 2024-03-16T10:10:55.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 -- 172.21.15.17:0/306437188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d30144870 con 0x7f1d301d52b0 2024-03-16T10:10:55.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d227fc640 1 -- 172.21.15.17:0/306437188 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c004020 con 0x7f1d301d52b0 2024-03-16T10:10:55.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 -- 172.21.15.17:0/306437188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d301d6480 con 0x7f1d301d52b0 2024-03-16T10:10:55.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.509+0000 7f1d38218640 1 -- 172.21.15.17:0/306437188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d30068360 con 0x7f1d301d52b0 2024-03-16T10:10:55.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.513+0000 7f1d227fc640 1 -- 172.21.15.17:0/306437188 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f1d2c0041e0 con 0x7f1d301d52b0 2024-03-16T10:10:55.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.513+0000 7f1d227fc640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1d14067660 0x7f1d14069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:55.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.513+0000 7f1d227fc640 1 -- 172.21.15.17:0/306437188 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f1d2c085940 con 0x7f1d301d52b0 2024-03-16T10:10:55.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.513+0000 7f1d35fb4640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1d14067660 0x7f1d14069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:55.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.513+0000 7f1d35fb4640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1d14067660 0x7f1d14069b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d240ae000 tx=0x7f1d240046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:55.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.517+0000 7f1d227fc640 1 -- 172.21.15.17:0/306437188 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1d2c018030 con 0x7f1d301d52b0 2024-03-16T10:10:55.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:55.677+0000 7f1d38218640 1 -- 172.21.15.17:0/306437188 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1d300b5710 con 0x7f1d301d52b0 2024-03-16T10:10:56.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.073+0000 7f1d227fc640 1 -- 172.21.15.17:0/306437188 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f1d2c056910 con 0x7f1d301d52b0 2024-03-16T10:10:56.077 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-16T10:10:56.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.077+0000 7f1d38218640 1 -- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1d14067660 msgr2=0x7f1d14069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:56.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.077+0000 7f1d38218640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1d14067660 0x7f1d14069b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1d240ae000 tx=0x7f1d240046d0 comp rx=0 tx=0).stop 2024-03-16T10:10:56.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.077+0000 7f1d38218640 1 -- 172.21.15.17:0/306437188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d301d52b0 msgr2=0x7f1d301e36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:56.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.077+0000 7f1d38218640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d301d52b0 0x7f1d301e36d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c010950 tx=0x7f1d2c010e10 comp rx=0 tx=0).stop 2024-03-16T10:10:56.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.077+0000 7f1d38218640 1 -- 172.21.15.17:0/306437188 shutdown_connections 2024-03-16T10:10:56.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.077+0000 7f1d38218640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1d14067660 0x7f1d14069b20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:56.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.077+0000 7f1d38218640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1d301e3c10 0x7f1d3013cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:56.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.077+0000 7f1d38218640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d301d52b0 0x7f1d301e36d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:56.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.077+0000 7f1d38218640 1 --2- 172.21.15.17:0/306437188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d1c0039c0 0x7f1d301d4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:56.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.077+0000 7f1d38218640 1 -- 172.21.15.17:0/306437188 >> 172.21.15.17:0/306437188 conn(0x7f1d300bccd0 msgr2=0x7f1d300b7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:56.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.077+0000 7f1d38218640 1 -- 172.21.15.17:0/306437188 shutdown_connections 2024-03-16T10:10:56.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.077+0000 7f1d38218640 1 -- 172.21.15.17:0/306437188 wait complete. 2024-03-16T10:10:56.106 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-16T10:10:56.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.237+0000 7f3f67054640 1 Processor -- start 2024-03-16T10:10:56.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.237+0000 7f3f67054640 1 -- start start 2024-03-16T10:10:56.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.237+0000 7f3f67054640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f600b58e0 0x7f3f6006f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:56.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.237+0000 7f3f67054640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f6006c0b0 con 0x7f3f6006ddc0 2024-03-16T10:10:56.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.237+0000 7f3f67054640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f600b5cc0 con 0x7f3f6006d9a0 2024-03-16T10:10:56.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.237+0000 7f3f67054640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3f600b0300 con 0x7f3f600b58e0 2024-03-16T10:10:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.237+0000 7f3f655f1640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3f6006ddc0 0x7f3f6006b9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47534/0 (socket says 172.21.15.17:47534) 2024-03-16T10:10:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.237+0000 7f3f655f1640 1 -- 172.21.15.17:0/405703711 learned_addr learned my addr 172.21.15.17:0/405703711 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:10:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.237+0000 7f3f64df0640 1 --2- 172.21.15.17:0/405703711 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f600b58e0 0x7f3f6006f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.237+0000 7f3f5f7fe640 1 -- 172.21.15.17:0/405703711 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2159450553 0 0) 0x7f3f6006c0b0 con 0x7f3f6006ddc0 2024-03-16T10:10:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.237+0000 7f3f5f7fe640 1 -- 172.21.15.17:0/405703711 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f48003530 con 0x7f3f6006ddc0 2024-03-16T10:10:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.237+0000 7f3f5f7fe640 1 -- 172.21.15.17:0/405703711 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2916668884 0 0) 0x7f3f48003530 con 0x7f3f6006ddc0 2024-03-16T10:10:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.237+0000 7f3f5f7fe640 1 -- 172.21.15.17:0/405703711 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f600b58e0 msgr2=0x7f3f6006f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.237+0000 7f3f5f7fe640 1 --2- 172.21.15.17:0/405703711 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f600b58e0 0x7f3f6006f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5f7fe640 1 -- 172.21.15.17:0/405703711 >> v1:172.21.15.174:6789/0 conn(0x7f3f6006d9a0 legacy=0x7f3f6006b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5f7fe640 1 -- 172.21.15.17:0/405703711 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6006d100 con 0x7f3f6006ddc0 2024-03-16T10:10:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5f7fe640 1 -- 172.21.15.17:0/405703711 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f3f58001a00 con 0x7f3f6006ddc0 2024-03-16T10:10:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5f7fe640 1 -- 172.21.15.17:0/405703711 >> v1:172.21.15.17:6789/0 conn(0x7f3f6006ddc0 legacy=0x7f3f6006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5f7fe640 1 --2- 172.21.15.17:0/405703711 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f600b58e0 0x7f3f480039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5f7fe640 1 --2- 172.21.15.17:0/405703711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f48003f70 0x7f3f48004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5f7fe640 1 --2- 172.21.15.17:0/405703711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f48004d50 0x7f3f480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5f7fe640 1 -- 172.21.15.17:0/405703711 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3f600b0300 con 0x7f3f48004d50 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5f7fe640 1 -- 172.21.15.17:0/405703711 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4800bb60 con 0x7f3f48003f70 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5f7fe640 1 -- 172.21.15.17:0/405703711 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4800bd60 con 0x7f3f600b58e0 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f655f1640 1 --2- 172.21.15.17:0/405703711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f48004d50 0x7f3f480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f64df0640 1 --2- 172.21.15.17:0/405703711 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f600b58e0 0x7f3f480039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5ffff640 1 --2- 172.21.15.17:0/405703711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f48003f70 0x7f3f48004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f655f1640 1 -- 172.21.15.17:0/405703711 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f600b58e0 msgr2=0x7f3f480039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f655f1640 1 --2- 172.21.15.17:0/405703711 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f600b58e0 0x7f3f480039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f655f1640 1 -- 172.21.15.17:0/405703711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f48003f70 msgr2=0x7f3f48004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f655f1640 1 --2- 172.21.15.17:0/405703711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f48003f70 0x7f3f48004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f655f1640 1 -- 172.21.15.17:0/405703711 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f48003530 con 0x7f3f48004d50 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5ffff640 1 --2- 172.21.15.17:0/405703711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f48003f70 0x7f3f48004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:10:56.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f655f1640 1 --2- 172.21.15.17:0/405703711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f48004d50 0x7f3f480051d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800a840 tx=0x7f3f5801fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e688e86ab2856cf server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5f7fe640 1 -- 172.21.15.17:0/405703711 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800a460 con 0x7f3f48004d50 2024-03-16T10:10:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5f7fe640 1 -- 172.21.15.17:0/405703711 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f580aa710 con 0x7f3f48004d50 2024-03-16T10:10:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f5f7fe640 1 -- 172.21.15.17:0/405703711 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f580aab00 con 0x7f3f48004d50 2024-03-16T10:10:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f67054640 1 -- 172.21.15.17:0/405703711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f48004d50 msgr2=0x7f3f480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f67054640 1 --2- 172.21.15.17:0/405703711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f48004d50 0x7f3f480051d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800a840 tx=0x7f3f5801fac0 comp rx=0 tx=0).stop 2024-03-16T10:10:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f655f1640 1 -- 172.21.15.17:0/405703711 reap_dead start 2024-03-16T10:10:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f67054640 1 -- 172.21.15.17:0/405703711 shutdown_connections 2024-03-16T10:10:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.241+0000 7f3f67054640 1 -- 172.21.15.17:0/405703711 >> 172.21.15.17:0/405703711 conn(0x7f3f6006c3a0 msgr2=0x7f3f6006c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f67054640 1 -- 172.21.15.17:0/405703711 shutdown_connections 2024-03-16T10:10:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f67054640 1 -- 172.21.15.17:0/405703711 wait complete. 2024-03-16T10:10:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f67054640 1 Processor -- start 2024-03-16T10:10:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f67054640 1 -- start start 2024-03-16T10:10:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f67054640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f48004d50 0x7f3f6014a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f67054640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f600b1620 0x7f3f601dfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f67054640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f60140c70 0x7f3f601410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f67054640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3f600afde0 con 0x7f3f60140c70 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f67054640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f600afb00 con 0x7f3f600b1620 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f67054640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3f6006d5d0 con 0x7f3f48004d50 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f655f1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f60140c70 0x7f3f601410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f655f1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f60140c70 0x7f3f601410f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43690/0 (socket says 172.21.15.17:43690) 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f655f1640 1 -- 172.21.15.17:0/1572481406 learned_addr learned my addr 172.21.15.17:0/1572481406 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f64df0640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f48004d50 0x7f3f6014a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f5ffff640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f600b1620 0x7f3f601dfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f655f1640 1 -- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f48004d50 msgr2=0x7f3f6014a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f655f1640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f48004d50 0x7f3f6014a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f655f1640 1 -- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f600b1620 msgr2=0x7f3f601dfdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f655f1640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f600b1620 0x7f3f601dfdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f655f1640 1 -- 172.21.15.17:0/1572481406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f600b3270 con 0x7f3f60140c70 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f5ffff640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f600b1620 0x7f3f601dfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:10:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f655f1640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f60140c70 0x7f3f601410f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3f58001d00 tx=0x7f3f48005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:56.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f5d7fa640 1 -- 172.21.15.17:0/1572481406 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f580b7070 con 0x7f3f60140c70 2024-03-16T10:10:56.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f5d7fa640 1 -- 172.21.15.17:0/1572481406 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f580aa8c0 con 0x7f3f60140c70 2024-03-16T10:10:56.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f5d7fa640 1 -- 172.21.15.17:0/1572481406 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800a460 con 0x7f3f60140c70 2024-03-16T10:10:56.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f67054640 1 -- 172.21.15.17:0/1572481406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f600b2940 con 0x7f3f60140c70 2024-03-16T10:10:56.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f67054640 1 -- 172.21.15.17:0/1572481406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f60142ef0 con 0x7f3f60140c70 2024-03-16T10:10:56.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.245+0000 7f3f67054640 1 -- 172.21.15.17:0/1572481406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f6006a130 con 0x7f3f60140c70 2024-03-16T10:10:56.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.249+0000 7f3f5d7fa640 1 -- 172.21.15.17:0/1572481406 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f3f580aac70 con 0x7f3f60140c70 2024-03-16T10:10:56.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.249+0000 7f3f5d7fa640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3f4c067660 0x7f3f4c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:56.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.249+0000 7f3f5d7fa640 1 -- 172.21.15.17:0/1572481406 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f3f58122c80 con 0x7f3f60140c70 2024-03-16T10:10:56.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.249+0000 7f3f64df0640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3f4c067660 0x7f3f4c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:56.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.249+0000 7f3f64df0640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3f4c067660 0x7f3f4c069b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009be0 tx=0x7f3f54009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.253+0000 7f3f5d7fa640 1 -- 172.21.15.17:0/1572481406 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3f580c3030 con 0x7f3f60140c70 2024-03-16T10:10:56.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:56.413+0000 7f3f67054640 1 -- 172.21.15.17:0/1572481406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3f600b7ba0 con 0x7f3f60140c70 2024-03-16T10:10:57.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.077+0000 7f3f5d7fa640 1 -- 172.21.15.17:0/1572481406 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7f3f580ae030 con 0x7f3f60140c70 2024-03-16T10:10:57.082 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-16T10:10:57.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.081+0000 7f3f67054640 1 -- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3f4c067660 msgr2=0x7f3f4c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:57.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.081+0000 7f3f67054640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3f4c067660 0x7f3f4c069b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3f54009be0 tx=0x7f3f54009190 comp rx=0 tx=0).stop 2024-03-16T10:10:57.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.081+0000 7f3f67054640 1 -- 172.21.15.17:0/1572481406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f60140c70 msgr2=0x7f3f601410f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:57.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.081+0000 7f3f67054640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f60140c70 0x7f3f601410f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3f58001d00 tx=0x7f3f48005840 comp rx=0 tx=0).stop 2024-03-16T10:10:57.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.081+0000 7f3f67054640 1 -- 172.21.15.17:0/1572481406 shutdown_connections 2024-03-16T10:10:57.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.081+0000 7f3f67054640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3f4c067660 0x7f3f4c069b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:57.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.081+0000 7f3f67054640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f60140c70 0x7f3f601410f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:57.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.081+0000 7f3f67054640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f600b1620 0x7f3f601dfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:57.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.081+0000 7f3f67054640 1 --2- 172.21.15.17:0/1572481406 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3f48004d50 0x7f3f6014a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:57.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.081+0000 7f3f67054640 1 -- 172.21.15.17:0/1572481406 >> 172.21.15.17:0/1572481406 conn(0x7f3f6006c3a0 msgr2=0x7f3f6006c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:57.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.081+0000 7f3f67054640 1 -- 172.21.15.17:0/1572481406 shutdown_connections 2024-03-16T10:10:57.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.081+0000 7f3f67054640 1 -- 172.21.15.17:0/1572481406 wait complete. 2024-03-16T10:10:57.126 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-16T10:10:57.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47cac18640 1 Processor -- start 2024-03-16T10:10:57.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47cac18640 1 -- start start 2024-03-16T10:10:57.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47cac18640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47c4072b10 0x7f47c4148740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:57.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47cac18640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f47c406f870 0x7f47c41422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:57.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47cac18640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47c4072ef0 con 0x7f47c406fd00 2024-03-16T10:10:57.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47cac18640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f47c40b0360 con 0x7f47c4072b10 2024-03-16T10:10:57.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47cac18640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f47c40b0c00 con 0x7f47c406f870 2024-03-16T10:10:57.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47c89b4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47c4072b10 0x7f47c4148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:57.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47c89b4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47c4072b10 0x7f47c4148740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43694/0 (socket says 172.21.15.17:43694) 2024-03-16T10:10:57.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47c89b4640 1 -- 172.21.15.17:0/3067251605 learned_addr learned my addr 172.21.15.17:0/3067251605 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:57.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47c3fff640 1 --2- 172.21.15.17:0/3067251605 >> v2:172.21.15.174:3300/0 conn(0x7f47c406f870 0x7f47c41422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:57.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47c89b4640 1 -- 172.21.15.17:0/3067251605 >> v2:172.21.15.174:3300/0 conn(0x7f47c406f870 msgr2=0x7f47c41422b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:57.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47c89b4640 1 --2- 172.21.15.17:0/3067251605 >> v2:172.21.15.174:3300/0 conn(0x7f47c406f870 0x7f47c41422b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:57.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47c89b4640 1 -- 172.21.15.17:0/3067251605 >> v1:172.21.15.174:6789/0 conn(0x7f47c406fd00 legacy=0x7f47c40b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:57.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.249+0000 7f47c89b4640 1 -- 172.21.15.17:0/3067251605 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c40b0ee0 con 0x7f47c4072b10 2024-03-16T10:10:57.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47c89b4640 1 --2- 172.21.15.17:0/3067251605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47c4072b10 0x7f47c4148740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f47c40b0820 tx=0x7f47b401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82106da600ddf4f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:57.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47c37fe640 1 -- 172.21.15.17:0/3067251605 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b40b0070 con 0x7f47c4072b10 2024-03-16T10:10:57.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47c37fe640 1 -- 172.21.15.17:0/3067251605 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b40ab600 con 0x7f47c4072b10 2024-03-16T10:10:57.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47c37fe640 1 -- 172.21.15.17:0/3067251605 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b40ab9d0 con 0x7f47c4072b10 2024-03-16T10:10:57.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 -- 172.21.15.17:0/3067251605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47c4072b10 msgr2=0x7f47c4148740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:57.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 --2- 172.21.15.17:0/3067251605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47c4072b10 0x7f47c4148740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f47c40b0820 tx=0x7f47b401c490 comp rx=0 tx=0).stop 2024-03-16T10:10:57.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 -- 172.21.15.17:0/3067251605 shutdown_connections 2024-03-16T10:10:57.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 --2- 172.21.15.17:0/3067251605 >> v2:172.21.15.174:3300/0 conn(0x7f47c406f870 0x7f47c41422b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:57.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 --2- 172.21.15.17:0/3067251605 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47c4072b10 0x7f47c4148740 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:57.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 -- 172.21.15.17:0/3067251605 >> 172.21.15.17:0/3067251605 conn(0x7f47c40b77f0 msgr2=0x7f47c40b7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:57.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 -- 172.21.15.17:0/3067251605 shutdown_connections 2024-03-16T10:10:57.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 -- 172.21.15.17:0/3067251605 wait complete. 2024-03-16T10:10:57.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 Processor -- start 2024-03-16T10:10:57.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 -- start start 2024-03-16T10:10:57.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47c406f870 0x7f47c41477f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:57.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47c406fd00 0x7f47c40b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:57.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47c40b44d0 0x7f47c4139e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:57.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4148e00 con 0x7f47c40b44d0 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47c4148c80 con 0x7f47c406f870 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47cac18640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f47c4152ab0 con 0x7f47c406fd00 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47c91b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47c40b44d0 0x7f47c4139e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47c91b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47c40b44d0 0x7f47c4139e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43698/0 (socket says 172.21.15.17:43698) 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47c91b5640 1 -- 172.21.15.17:0/2061628404 learned_addr learned my addr 172.21.15.17:0/2061628404 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47c89b4640 1 --2- 172.21.15.17:0/2061628404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47c406f870 0x7f47c41477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47c3fff640 1 --2- 172.21.15.17:0/2061628404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47c406fd00 0x7f47c40b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47c91b5640 1 -- 172.21.15.17:0/2061628404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47c406fd00 msgr2=0x7f47c40b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47c91b5640 1 --2- 172.21.15.17:0/2061628404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47c406fd00 0x7f47c40b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47c91b5640 1 -- 172.21.15.17:0/2061628404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47c406f870 msgr2=0x7f47c41477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47c91b5640 1 --2- 172.21.15.17:0/2061628404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47c406f870 0x7f47c41477f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.253+0000 7f47c91b5640 1 -- 172.21.15.17:0/2061628404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c413fb70 con 0x7f47c40b44d0 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.257+0000 7f47c91b5640 1 --2- 172.21.15.17:0/2061628404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47c40b44d0 0x7f47c4139e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c730 tx=0x7f47bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.257+0000 7f47c17fa640 1 -- 172.21.15.17:0/2061628404 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc003b80 con 0x7f47c40b44d0 2024-03-16T10:10:57.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.257+0000 7f47c17fa640 1 -- 172.21.15.17:0/2061628404 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47bc003d20 con 0x7f47c40b44d0 2024-03-16T10:10:57.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.257+0000 7f47cac18640 1 -- 172.21.15.17:0/2061628404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c40b20f0 con 0x7f47c40b44d0 2024-03-16T10:10:57.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.257+0000 7f47c17fa640 1 -- 172.21.15.17:0/2061628404 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc005310 con 0x7f47c40b44d0 2024-03-16T10:10:57.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.257+0000 7f47cac18640 1 -- 172.21.15.17:0/2061628404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c4147f90 con 0x7f47c40b44d0 2024-03-16T10:10:57.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.257+0000 7f47cac18640 1 -- 172.21.15.17:0/2061628404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c40b22c0 con 0x7f47c40b44d0 2024-03-16T10:10:57.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.257+0000 7f47c17fa640 1 -- 172.21.15.17:0/2061628404 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f47bc0056c0 con 0x7f47c40b44d0 2024-03-16T10:10:57.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.257+0000 7f47c17fa640 1 --2- 172.21.15.17:0/2061628404 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f47b0067660 0x7f47b0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:57.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.257+0000 7f47c17fa640 1 -- 172.21.15.17:0/2061628404 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f47bc0857a0 con 0x7f47c40b44d0 2024-03-16T10:10:57.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.257+0000 7f47c89b4640 1 --2- 172.21.15.17:0/2061628404 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f47b0067660 0x7f47b0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:57.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.257+0000 7f47c89b4640 1 --2- 172.21.15.17:0/2061628404 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f47b0067660 0x7f47b0069b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47b40ad000 tx=0x7f47b4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:57.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.261+0000 7f47c17fa640 1 -- 172.21.15.17:0/2061628404 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f47bc056510 con 0x7f47c40b44d0 2024-03-16T10:10:57.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:57.457+0000 7f47cac18640 1 -- 172.21.15.17:0/2061628404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f47c4146110 con 0x7f47c40b44d0 2024-03-16T10:10:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.137+0000 7f47c17fa640 1 -- 172.21.15.17:0/2061628404 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f47bc055e10 con 0x7f47c40b44d0 2024-03-16T10:10:58.146 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 2 and data pool 3 2024-03-16T10:10:58.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.145+0000 7f47cac18640 1 -- 172.21.15.17:0/2061628404 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f47b0067660 msgr2=0x7f47b0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.145+0000 7f47cac18640 1 --2- 172.21.15.17:0/2061628404 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f47b0067660 0x7f47b0069b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47b40ad000 tx=0x7f47b4008040 comp rx=0 tx=0).stop 2024-03-16T10:10:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.145+0000 7f47cac18640 1 -- 172.21.15.17:0/2061628404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47c40b44d0 msgr2=0x7f47c4139e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.145+0000 7f47cac18640 1 --2- 172.21.15.17:0/2061628404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47c40b44d0 0x7f47c4139e20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c730 tx=0x7f47bc00cbf0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.145+0000 7f47cac18640 1 -- 172.21.15.17:0/2061628404 shutdown_connections 2024-03-16T10:10:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.145+0000 7f47cac18640 1 --2- 172.21.15.17:0/2061628404 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f47b0067660 0x7f47b0069b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.145+0000 7f47cac18640 1 --2- 172.21.15.17:0/2061628404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47c40b44d0 0x7f47c4139e20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.145+0000 7f47cac18640 1 --2- 172.21.15.17:0/2061628404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47c406fd00 0x7f47c40b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.145+0000 7f47cac18640 1 --2- 172.21.15.17:0/2061628404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47c406f870 0x7f47c41477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.145+0000 7f47cac18640 1 -- 172.21.15.17:0/2061628404 >> 172.21.15.17:0/2061628404 conn(0x7f47c40b77f0 msgr2=0x7f47c4143140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.149+0000 7f47cac18640 1 -- 172.21.15.17:0/2061628404 shutdown_connections 2024-03-16T10:10:58.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.149+0000 7f47cac18640 1 -- 172.21.15.17:0/2061628404 wait complete. 2024-03-16T10:10:58.180 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:10:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd80584c640 1 Processor -- start 2024-03-16T10:10:58.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd80584c640 1 -- start start 2024-03-16T10:10:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd80584c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd80006d750 0x7fd80006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd80584c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd800145280 0x7fd80013f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd80584c640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd80006db30 con 0x7fd8000b2820 2024-03-16T10:10:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd80584c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd8000b2d30 con 0x7fd800145280 2024-03-16T10:10:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd80584c640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fd80006fcf0 con 0x7fd80006d750 2024-03-16T10:10:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd7ff7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd800145280 0x7fd80013f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd7ff7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd800145280 0x7fd80013f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43708/0 (socket says 172.21.15.17:43708) 2024-03-16T10:10:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd7ff7fe640 1 -- 172.21.15.17:0/184315451 learned_addr learned my addr 172.21.15.17:0/184315451 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd7feffd640 1 --2- 172.21.15.17:0/184315451 >> v2:172.21.15.174:3300/0 conn(0x7fd80006d750 0x7fd80006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd7ff7fe640 1 -- 172.21.15.17:0/184315451 >> v2:172.21.15.174:3300/0 conn(0x7fd80006d750 msgr2=0x7fd80006f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd7ff7fe640 1 --2- 172.21.15.17:0/184315451 >> v2:172.21.15.174:3300/0 conn(0x7fd80006d750 0x7fd80006f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd7ff7fe640 1 -- 172.21.15.17:0/184315451 >> v1:172.21.15.174:6789/0 conn(0x7fd8000b2820 legacy=0x7fd80013eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:58.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.293+0000 7fd7ff7fe640 1 -- 172.21.15.17:0/184315451 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8000afaf0 con 0x7fd800145280 2024-03-16T10:10:58.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.297+0000 7fd7ff7fe640 1 --2- 172.21.15.17:0/184315451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd800145280 0x7fd80013f5f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd80006de30 tx=0x7fd7f801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363dbdb245ebe809 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:58.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.297+0000 7fd7fdffb640 1 -- 172.21.15.17:0/184315451 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80a5550 con 0x7fd800145280 2024-03-16T10:10:58.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.297+0000 7fd7fdffb640 1 -- 172.21.15.17:0/184315451 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f80a56f0 con 0x7fd800145280 2024-03-16T10:10:58.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.297+0000 7fd7fdffb640 1 -- 172.21.15.17:0/184315451 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80a5ac0 con 0x7fd800145280 2024-03-16T10:10:58.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.297+0000 7fd80584c640 1 -- 172.21.15.17:0/184315451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd800145280 msgr2=0x7fd80013f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.297+0000 7fd80584c640 1 --2- 172.21.15.17:0/184315451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd800145280 0x7fd80013f5f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd80006de30 tx=0x7fd7f801c890 comp rx=0 tx=0).stop 2024-03-16T10:10:58.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.297+0000 7fd80584c640 1 -- 172.21.15.17:0/184315451 shutdown_connections 2024-03-16T10:10:58.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.297+0000 7fd80584c640 1 --2- 172.21.15.17:0/184315451 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd800145280 0x7fd80013f5f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.297+0000 7fd80584c640 1 --2- 172.21.15.17:0/184315451 >> v2:172.21.15.174:3300/0 conn(0x7fd80006d750 0x7fd80006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.297+0000 7fd80584c640 1 -- 172.21.15.17:0/184315451 >> 172.21.15.17:0/184315451 conn(0x7fd8000b55c0 msgr2=0x7fd8000b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:58.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.297+0000 7fd80584c640 1 -- 172.21.15.17:0/184315451 shutdown_connections 2024-03-16T10:10:58.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.297+0000 7fd80584c640 1 -- 172.21.15.17:0/184315451 wait complete. 2024-03-16T10:10:58.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.297+0000 7fd80584c640 1 Processor -- start 2024-03-16T10:10:58.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.297+0000 7fd80584c640 1 -- start start 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd80584c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd80006d750 0x7fd800139590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd80584c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8000b2820 0x7fd800140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd80584c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd800145280 0x7fd8000712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd80584c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd800152550 con 0x7fd800145280 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd80584c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8000b49f0 con 0x7fd8000b2820 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd80584c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd8001526d0 con 0x7fd80006d750 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7ff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd800145280 0x7fd8000712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7ff7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd800145280 0x7fd8000712d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43710/0 (socket says 172.21.15.17:43710) 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7ff7fe640 1 -- 172.21.15.17:0/2821741264 learned_addr learned my addr 172.21.15.17:0/2821741264 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7feffd640 1 --2- 172.21.15.17:0/2821741264 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd80006d750 0x7fd800139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7fe7fc640 1 --2- 172.21.15.17:0/2821741264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8000b2820 0x7fd800140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7ff7fe640 1 -- 172.21.15.17:0/2821741264 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd80006d750 msgr2=0x7fd800139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7ff7fe640 1 --2- 172.21.15.17:0/2821741264 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd80006d750 0x7fd800139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7ff7fe640 1 -- 172.21.15.17:0/2821741264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8000b2820 msgr2=0x7fd800140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7ff7fe640 1 --2- 172.21.15.17:0/2821741264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8000b2820 0x7fd800140720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7ff7fe640 1 -- 172.21.15.17:0/2821741264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80013fc90 con 0x7fd800145280 2024-03-16T10:10:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7ff7fe640 1 --2- 172.21.15.17:0/2821741264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd800145280 0x7fd8000712d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80af040 tx=0x7fd7f8019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7e7fff640 1 -- 172.21.15.17:0/2821741264 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80bc020 con 0x7fd800145280 2024-03-16T10:10:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7e7fff640 1 -- 172.21.15.17:0/2821741264 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f80b8880 con 0x7fd800145280 2024-03-16T10:10:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd80584c640 1 -- 172.21.15.17:0/2821741264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8001425f0 con 0x7fd800145280 2024-03-16T10:10:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7e7fff640 1 -- 172.21.15.17:0/2821741264 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80a5550 con 0x7fd800145280 2024-03-16T10:10:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd80584c640 1 -- 172.21.15.17:0/2821741264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd800142a80 con 0x7fd800145280 2024-03-16T10:10:58.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd80584c640 1 -- 172.21.15.17:0/2821741264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd800068360 con 0x7fd800145280 2024-03-16T10:10:58.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7e7fff640 1 -- 172.21.15.17:0/2821741264 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd7f80bf020 con 0x7fd800145280 2024-03-16T10:10:58.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7e7fff640 1 --2- 172.21.15.17:0/2821741264 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd7ec067690 0x7fd7ec069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7e7fff640 1 -- 172.21.15.17:0/2821741264 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd7f8120600 con 0x7fd800145280 2024-03-16T10:10:58.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.301+0000 7fd7feffd640 1 --2- 172.21.15.17:0/2821741264 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd7ec067690 0x7fd7ec069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.305+0000 7fd7feffd640 1 --2- 172.21.15.17:0/2821741264 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd7ec067690 0x7fd7ec069b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400b9d0 tx=0x7fd7f4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:58.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.305+0000 7fd7e7fff640 1 -- 172.21.15.17:0/2821741264 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd7f80c6070 con 0x7fd800145280 2024-03-16T10:10:58.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.469+0000 7fd80584c640 1 -- 172.21.15.17:0/2821741264 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd800142310 con 0x7fd800145280 2024-03-16T10:10:58.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.469+0000 7fd7e7fff640 1 -- 172.21.15.17:0/2821741264 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16618 (secure 0 0 0) 0x7fd7f80165a0 con 0x7fd800145280 2024-03-16T10:10:58.475 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:58.475 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":20,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:57.463957+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:10:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.473+0000 7fd80584c640 1 -- 172.21.15.17:0/2821741264 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd7ec067690 msgr2=0x7fd7ec069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.473+0000 7fd80584c640 1 --2- 172.21.15.17:0/2821741264 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd7ec067690 0x7fd7ec069b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400b9d0 tx=0x7fd7f4008040 comp rx=0 tx=0).stop 2024-03-16T10:10:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.473+0000 7fd80584c640 1 -- 172.21.15.17:0/2821741264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd800145280 msgr2=0x7fd8000712d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.473+0000 7fd80584c640 1 --2- 172.21.15.17:0/2821741264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd800145280 0x7fd8000712d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80af040 tx=0x7fd7f8019630 comp rx=0 tx=0).stop 2024-03-16T10:10:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.473+0000 7fd80584c640 1 -- 172.21.15.17:0/2821741264 shutdown_connections 2024-03-16T10:10:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.473+0000 7fd80584c640 1 --2- 172.21.15.17:0/2821741264 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd7ec067690 0x7fd7ec069b50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.473+0000 7fd80584c640 1 --2- 172.21.15.17:0/2821741264 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd800145280 0x7fd8000712d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.473+0000 7fd80584c640 1 --2- 172.21.15.17:0/2821741264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8000b2820 0x7fd800140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.473+0000 7fd80584c640 1 --2- 172.21.15.17:0/2821741264 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd80006d750 0x7fd800139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.473+0000 7fd80584c640 1 -- 172.21.15.17:0/2821741264 >> 172.21.15.17:0/2821741264 conn(0x7fd8000b55c0 msgr2=0x7fd80013a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:58.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.473+0000 7fd80584c640 1 -- 172.21.15.17:0/2821741264 shutdown_connections 2024-03-16T10:10:58.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.473+0000 7fd80584c640 1 -- 172.21.15.17:0/2821741264 wait complete. 2024-03-16T10:10:58.495 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:10:58.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.617+0000 7fd05a544640 1 Processor -- start 2024-03-16T10:10:58.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.617+0000 7fd05a544640 1 -- start start 2024-03-16T10:10:58.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd05a544640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd054145280 0x7fd05413f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd05a544640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd05406db30 con 0x7fd0540b2820 2024-03-16T10:10:58.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd05a544640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0540b2d30 con 0x7fd05406d750 2024-03-16T10:10:58.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd05a544640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd0540afaf0 con 0x7fd054145280 2024-03-16T10:10:58.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd0537fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd0540b2820 0x7fd05413eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:47544/0 (socket says 172.21.15.17:47544) 2024-03-16T10:10:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd0537fe640 1 -- 172.21.15.17:0/3278230887 learned_addr learned my addr 172.21.15.17:0/3278230887 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:10:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd058ae1640 1 --2- 172.21.15.17:0/3278230887 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd054145280 0x7fd05413f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 -- 172.21.15.17:0/3278230887 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3240906007 0 0) 0x7fd05406db30 con 0x7fd0540b2820 2024-03-16T10:10:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 -- 172.21.15.17:0/3278230887 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd03c003530 con 0x7fd0540b2820 2024-03-16T10:10:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 -- 172.21.15.17:0/3278230887 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3457519523 0 0) 0x7fd03c003530 con 0x7fd0540b2820 2024-03-16T10:10:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 -- 172.21.15.17:0/3278230887 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd054145280 msgr2=0x7fd05413f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 --2- 172.21.15.17:0/3278230887 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd054145280 0x7fd05413f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 -- 172.21.15.17:0/3278230887 >> v1:172.21.15.174:6789/0 conn(0x7fd05406d750 legacy=0x7fd05406f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 -- 172.21.15.17:0/3278230887 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0540afc70 con 0x7fd0540b2820 2024-03-16T10:10:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 -- 172.21.15.17:0/3278230887 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fd048001680 con 0x7fd0540b2820 2024-03-16T10:10:58.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 -- 172.21.15.17:0/3278230887 >> v1:172.21.15.17:6789/0 conn(0x7fd0540b2820 legacy=0x7fd05413eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 --2- 172.21.15.17:0/3278230887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd03c0039f0 0x7fd03c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 --2- 172.21.15.17:0/3278230887 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd054145280 0x7fd03c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 --2- 172.21.15.17:0/3278230887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd03c004d50 0x7fd03c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 -- 172.21.15.17:0/3278230887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd0540afaf0 con 0x7fd03c0039f0 2024-03-16T10:10:58.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 -- 172.21.15.17:0/3278230887 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c00bb90 con 0x7fd03c004d50 2024-03-16T10:10:58.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd052ffd640 1 -- 172.21.15.17:0/3278230887 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd03c00bd90 con 0x7fd054145280 2024-03-16T10:10:58.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd053fff640 1 --2- 172.21.15.17:0/3278230887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd03c0039f0 0x7fd03c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd0537fe640 1 --2- 172.21.15.17:0/3278230887 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd054145280 0x7fd03c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd053fff640 1 -- 172.21.15.17:0/3278230887 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd054145280 msgr2=0x7fd03c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:58.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd058ae1640 1 --2- 172.21.15.17:0/3278230887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd03c004d50 0x7fd03c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd053fff640 1 --2- 172.21.15.17:0/3278230887 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd054145280 0x7fd03c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd053fff640 1 -- 172.21.15.17:0/3278230887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd03c004d50 msgr2=0x7fd03c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:58.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd053fff640 1 --2- 172.21.15.17:0/3278230887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd03c004d50 0x7fd03c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd053fff640 1 -- 172.21.15.17:0/3278230887 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c003530 con 0x7fd03c0039f0 2024-03-16T10:10:58.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd058ae1640 1 --2- 172.21.15.17:0/3278230887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd03c004d50 0x7fd03c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:10:58.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.621+0000 7fd0537fe640 1 --2- 172.21.15.17:0/3278230887 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd054145280 0x7fd03c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:10:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd053fff640 1 --2- 172.21.15.17:0/3278230887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd03c0039f0 0x7fd03c003e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd04c001e70 tx=0x7fd04c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc57074a3a1d4cb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:58.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd052ffd640 1 -- 172.21.15.17:0/3278230887 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c0af070 con 0x7fd03c0039f0 2024-03-16T10:10:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd052ffd640 1 -- 172.21.15.17:0/3278230887 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd04c003b80 con 0x7fd03c0039f0 2024-03-16T10:10:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd052ffd640 1 -- 172.21.15.17:0/3278230887 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c001a90 con 0x7fd03c0039f0 2024-03-16T10:10:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd05a544640 1 -- 172.21.15.17:0/3278230887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd03c0039f0 msgr2=0x7fd03c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd05a544640 1 --2- 172.21.15.17:0/3278230887 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd03c0039f0 0x7fd03c003e10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd04c001e70 tx=0x7fd04c01c780 comp rx=0 tx=0).stop 2024-03-16T10:10:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd058ae1640 1 -- 172.21.15.17:0/3278230887 reap_dead start 2024-03-16T10:10:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd05a544640 1 -- 172.21.15.17:0/3278230887 shutdown_connections 2024-03-16T10:10:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd05a544640 1 -- 172.21.15.17:0/3278230887 >> 172.21.15.17:0/3278230887 conn(0x7fd0540b55c0 msgr2=0x7fd0540b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd05a544640 1 -- 172.21.15.17:0/3278230887 shutdown_connections 2024-03-16T10:10:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd05a544640 1 -- 172.21.15.17:0/3278230887 wait complete. 2024-03-16T10:10:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd05a544640 1 Processor -- start 2024-03-16T10:10:58.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd05a544640 1 -- start start 2024-03-16T10:10:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd05a544640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd03c0039f0 0x7fd0541e2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd05a544640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0541e34c0 0x7fd0541398c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd05a544640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd054139e00 0x7fd05413a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd05a544640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd054151550 con 0x7fd054139e00 2024-03-16T10:10:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd05a544640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd05413fa00 con 0x7fd03c0039f0 2024-03-16T10:10:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd05a544640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd0541516d0 con 0x7fd0541e34c0 2024-03-16T10:10:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd058ae1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd054139e00 0x7fd05413a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd058ae1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd054139e00 0x7fd05413a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43736/0 (socket says 172.21.15.17:43736) 2024-03-16T10:10:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd058ae1640 1 -- 172.21.15.17:0/149532869 learned_addr learned my addr 172.21.15.17:0/149532869 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd058ae1640 1 -- 172.21.15.17:0/149532869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0541e34c0 msgr2=0x7fd0541398c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd058ae1640 1 --2- 172.21.15.17:0/149532869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0541e34c0 0x7fd0541398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd058ae1640 1 -- 172.21.15.17:0/149532869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd03c0039f0 msgr2=0x7fd0541e2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.625+0000 7fd053fff640 1 --2- 172.21.15.17:0/149532869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd03c0039f0 0x7fd0541e2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.629+0000 7fd058ae1640 1 --2- 172.21.15.17:0/149532869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd03c0039f0 0x7fd0541e2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.629+0000 7fd058ae1640 1 -- 172.21.15.17:0/149532869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0540b1c30 con 0x7fd054139e00 2024-03-16T10:10:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.629+0000 7fd058ae1640 1 --2- 172.21.15.17:0/149532869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd054139e00 0x7fd05413a280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd054070750 tx=0x7fd044010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.629+0000 7fd050ff9640 1 -- 172.21.15.17:0/149532869 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04400f040 con 0x7fd054139e00 2024-03-16T10:10:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.629+0000 7fd050ff9640 1 -- 172.21.15.17:0/149532869 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd04400e8a0 con 0x7fd054139e00 2024-03-16T10:10:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.629+0000 7fd050ff9640 1 -- 172.21.15.17:0/149532869 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04400ec70 con 0x7fd054139e00 2024-03-16T10:10:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.629+0000 7fd05a544640 1 -- 172.21.15.17:0/149532869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd054140e50 con 0x7fd054139e00 2024-03-16T10:10:58.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.629+0000 7fd05a544640 1 -- 172.21.15.17:0/149532869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0540af2a0 con 0x7fd054139e00 2024-03-16T10:10:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.629+0000 7fd05a544640 1 -- 172.21.15.17:0/149532869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd054068360 con 0x7fd054139e00 2024-03-16T10:10:58.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.629+0000 7fd050ff9640 1 -- 172.21.15.17:0/149532869 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd044026420 con 0x7fd054139e00 2024-03-16T10:10:58.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.629+0000 7fd050ff9640 1 --2- 172.21.15.17:0/149532869 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd040067660 0x7fd040069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.629+0000 7fd050ff9640 1 -- 172.21.15.17:0/149532869 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd044085b50 con 0x7fd054139e00 2024-03-16T10:10:58.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.629+0000 7fd053fff640 1 --2- 172.21.15.17:0/149532869 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd040067660 0x7fd040069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.633+0000 7fd053fff640 1 --2- 172.21.15.17:0/149532869 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd040067660 0x7fd040069b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0ac000 tx=0x7fd04c001520 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:58.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.633+0000 7fd050ff9640 1 -- 172.21.15.17:0/149532869 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd044018030 con 0x7fd054139e00 2024-03-16T10:10:58.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.797+0000 7fd05a544640 1 -- 172.21.15.17:0/149532869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd0540b1e00 con 0x7fd054139e00 2024-03-16T10:10:58.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.801+0000 7fd050ff9640 1 -- 172.21.15.17:0/149532869 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16618 (secure 0 0 0) 0x7fd04405c090 con 0x7fd054139e00 2024-03-16T10:10:58.803 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:10:58.804 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":20,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:57.463957+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:10:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.801+0000 7fd05a544640 1 -- 172.21.15.17:0/149532869 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd040067660 msgr2=0x7fd040069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.801+0000 7fd05a544640 1 --2- 172.21.15.17:0/149532869 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd040067660 0x7fd040069b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd04c0ac000 tx=0x7fd04c001520 comp rx=0 tx=0).stop 2024-03-16T10:10:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.801+0000 7fd05a544640 1 -- 172.21.15.17:0/149532869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd054139e00 msgr2=0x7fd05413a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.801+0000 7fd05a544640 1 --2- 172.21.15.17:0/149532869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd054139e00 0x7fd05413a280 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd054070750 tx=0x7fd044010c20 comp rx=0 tx=0).stop 2024-03-16T10:10:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.801+0000 7fd05a544640 1 -- 172.21.15.17:0/149532869 shutdown_connections 2024-03-16T10:10:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.801+0000 7fd05a544640 1 --2- 172.21.15.17:0/149532869 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd040067660 0x7fd040069b20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.801+0000 7fd05a544640 1 --2- 172.21.15.17:0/149532869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd054139e00 0x7fd05413a280 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.801+0000 7fd05a544640 1 --2- 172.21.15.17:0/149532869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0541e34c0 0x7fd0541398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.801+0000 7fd05a544640 1 --2- 172.21.15.17:0/149532869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd03c0039f0 0x7fd0541e2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.801+0000 7fd05a544640 1 -- 172.21.15.17:0/149532869 >> 172.21.15.17:0/149532869 conn(0x7fd0540b55c0 msgr2=0x7fd0540b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.801+0000 7fd05a544640 1 -- 172.21.15.17:0/149532869 shutdown_connections 2024-03-16T10:10:58.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.801+0000 7fd05a544640 1 -- 172.21.15.17:0/149532869 wait complete. 2024-03-16T10:10:58.823 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-16T10:10:58.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c485f1640 1 Processor -- start 2024-03-16T10:10:58.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c485f1640 1 -- start start 2024-03-16T10:10:58.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c485f1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c40072b10 0x7f6c40148740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c485f1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f6c4006f870 0x7f6c401422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c485f1640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c40072ef0 con 0x7f6c4006fcd0 2024-03-16T10:10:58.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c485f1640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f6c400b0360 con 0x7f6c4006f870 2024-03-16T10:10:58.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c485f1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f6c400b0c00 con 0x7f6c40072b10 2024-03-16T10:10:58.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c45b8c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f6c4006f870 0x7f6c401422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c4638d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c40072b10 0x7f6c40148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c45b8c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f6c4006f870 0x7f6c401422b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.17:58064/0 (socket says 172.21.15.17:58064) 2024-03-16T10:10:58.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c46b8e640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f6c4006fcd0 0x7f6c400b56f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.17:55956/0 (socket says 172.21.15.17:55956) 2024-03-16T10:10:58.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c45b8c640 1 -- 172.21.15.17:0/2065639058 learned_addr learned my addr 172.21.15.17:0/2065639058 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:58.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c45b8c640 1 -- 172.21.15.17:0/2065639058 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c40072b10 msgr2=0x7f6c40148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:10:58.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c45b8c640 1 --2- 172.21.15.17:0/2065639058 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c40072b10 0x7f6c40148740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c45b8c640 1 -- 172.21.15.17:0/2065639058 >> v1:172.21.15.174:6789/0 conn(0x7f6c4006fcd0 legacy=0x7f6c400b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.949+0000 7f6c45b8c640 1 -- 172.21.15.17:0/2065639058 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c400b0ee0 con 0x7f6c4006f870 2024-03-16T10:10:58.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c45b8c640 1 --2- 172.21.15.17:0/2065639058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4006f870 0x7f6c401422b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009870 tx=0x7f6c3001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=465ad70bd24e5a96 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:58.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c4538b640 1 -- 172.21.15.17:0/2065639058 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c300b0070 con 0x7f6c4006f870 2024-03-16T10:10:58.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c4538b640 1 -- 172.21.15.17:0/2065639058 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c300ab600 con 0x7f6c4006f870 2024-03-16T10:10:58.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c4538b640 1 -- 172.21.15.17:0/2065639058 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c300ab9d0 con 0x7f6c4006f870 2024-03-16T10:10:58.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 -- 172.21.15.17:0/2065639058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4006f870 msgr2=0x7f6c401422b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 --2- 172.21.15.17:0/2065639058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4006f870 0x7f6c401422b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009870 tx=0x7f6c3001c4d0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 -- 172.21.15.17:0/2065639058 shutdown_connections 2024-03-16T10:10:58.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 --2- 172.21.15.17:0/2065639058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4006f870 0x7f6c401422b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 --2- 172.21.15.17:0/2065639058 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c40072b10 0x7f6c40148740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 -- 172.21.15.17:0/2065639058 >> 172.21.15.17:0/2065639058 conn(0x7f6c400b77f0 msgr2=0x7f6c400b7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:10:58.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 -- 172.21.15.17:0/2065639058 shutdown_connections 2024-03-16T10:10:58.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 -- 172.21.15.17:0/2065639058 wait complete. 2024-03-16T10:10:58.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 Processor -- start 2024-03-16T10:10:58.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 -- start start 2024-03-16T10:10:58.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c4006f870 0x7f6c4013b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4006fcd0 0x7f6c40144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c40072b10 0x7f6c401e0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40148e00 con 0x7f6c4006f870 2024-03-16T10:10:58.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c40148c80 con 0x7f6c4006fcd0 2024-03-16T10:10:58.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c485f1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f6c40152ab0 con 0x7f6c40072b10 2024-03-16T10:10:58.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c4638d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c4006f870 0x7f6c4013b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c4638d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c4006f870 0x7f6c4013b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43750/0 (socket says 172.21.15.17:43750) 2024-03-16T10:10:58.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.953+0000 7f6c4638d640 1 -- 172.21.15.17:0/2964086681 learned_addr learned my addr 172.21.15.17:0/2964086681 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:10:58.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c46b8e640 1 --2- 172.21.15.17:0/2964086681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c40072b10 0x7f6c401e0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c45b8c640 1 --2- 172.21.15.17:0/2964086681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4006fcd0 0x7f6c40144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c4638d640 1 -- 172.21.15.17:0/2964086681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c40072b10 msgr2=0x7f6c401e0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c4638d640 1 --2- 172.21.15.17:0/2964086681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c40072b10 0x7f6c401e0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c4638d640 1 -- 172.21.15.17:0/2964086681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4006fcd0 msgr2=0x7f6c40144a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:10:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c4638d640 1 --2- 172.21.15.17:0/2964086681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4006fcd0 0x7f6c40144a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:10:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c4638d640 1 -- 172.21.15.17:0/2964086681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c40144fc0 con 0x7f6c4006f870 2024-03-16T10:10:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c4638d640 1 --2- 172.21.15.17:0/2964086681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c4006f870 0x7f6c4013b260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00e870 tx=0x7f6c3c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c36ffd640 1 -- 172.21.15.17:0/2964086681 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c017070 con 0x7f6c4006f870 2024-03-16T10:10:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c36ffd640 1 -- 172.21.15.17:0/2964086681 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c3c00c940 con 0x7f6c4006f870 2024-03-16T10:10:58.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c485f1640 1 -- 172.21.15.17:0/2964086681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4013bfc0 con 0x7f6c4006f870 2024-03-16T10:10:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c36ffd640 1 -- 172.21.15.17:0/2964086681 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c00ccf0 con 0x7f6c4006f870 2024-03-16T10:10:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c485f1640 1 -- 172.21.15.17:0/2964086681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c40145520 con 0x7f6c4006f870 2024-03-16T10:10:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c485f1640 1 -- 172.21.15.17:0/2964086681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c4006a130 con 0x7f6c4006f870 2024-03-16T10:10:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c36ffd640 1 -- 172.21.15.17:0/2964086681 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f6c3c01a070 con 0x7f6c4006f870 2024-03-16T10:10:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c36ffd640 1 --2- 172.21.15.17:0/2964086681 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6c24067660 0x7f6c24069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:10:58.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c36ffd640 1 -- 172.21.15.17:0/2964086681 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6c3c0856b0 con 0x7f6c4006f870 2024-03-16T10:10:58.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c45b8c640 1 --2- 172.21.15.17:0/2964086681 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6c24067660 0x7f6c24069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:10:58.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.957+0000 7f6c45b8c640 1 --2- 172.21.15.17:0/2964086681 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6c24067660 0x7f6c24069b20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6c300ad000 tx=0x7f6c30008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:10:58.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:58.961+0000 7f6c36ffd640 1 -- 172.21.15.17:0/2964086681 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6c40144fc0 con 0x7f6c4006f870 2024-03-16T10:10:59.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:10:59.161+0000 7f6c485f1640 1 -- 172.21.15.17:0/2964086681 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6c40145c50 con 0x7f6c4006f870 2024-03-16T10:10:59.689 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T10:10:59.692 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T10:11:00.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.141+0000 7f6c36ffd640 1 -- 172.21.15.17:0/2964086681 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f6c3c0563c0 con 0x7f6c4006f870 2024-03-16T10:11:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.145+0000 7f6c485f1640 1 -- 172.21.15.17:0/2964086681 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6c24067660 msgr2=0x7f6c24069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.145+0000 7f6c485f1640 1 --2- 172.21.15.17:0/2964086681 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6c24067660 0x7f6c24069b20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6c300ad000 tx=0x7f6c30008040 comp rx=0 tx=0).stop 2024-03-16T10:11:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.145+0000 7f6c485f1640 1 -- 172.21.15.17:0/2964086681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c4006f870 msgr2=0x7f6c4013b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.145+0000 7f6c485f1640 1 --2- 172.21.15.17:0/2964086681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c4006f870 0x7f6c4013b260 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00e870 tx=0x7f6c3c00ec70 comp rx=0 tx=0).stop 2024-03-16T10:11:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.145+0000 7f6c485f1640 1 -- 172.21.15.17:0/2964086681 shutdown_connections 2024-03-16T10:11:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.145+0000 7f6c485f1640 1 --2- 172.21.15.17:0/2964086681 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6c24067660 0x7f6c24069b20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.145+0000 7f6c485f1640 1 --2- 172.21.15.17:0/2964086681 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6c40072b10 0x7f6c401e0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.145+0000 7f6c485f1640 1 --2- 172.21.15.17:0/2964086681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4006fcd0 0x7f6c40144a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.145+0000 7f6c485f1640 1 --2- 172.21.15.17:0/2964086681 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c4006f870 0x7f6c4013b260 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.145+0000 7f6c485f1640 1 -- 172.21.15.17:0/2964086681 >> 172.21.15.17:0/2964086681 conn(0x7f6c400b77f0 msgr2=0x7f6c4006edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.145+0000 7f6c485f1640 1 -- 172.21.15.17:0/2964086681 shutdown_connections 2024-03-16T10:11:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.145+0000 7f6c485f1640 1 -- 172.21.15.17:0/2964086681 wait complete. 2024-03-16T10:11:00.169 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-16T10:11:00.169 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-16T10:11:00.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d897f640 1 Processor -- start 2024-03-16T10:11:00.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d897f640 1 -- start start 2024-03-16T10:11:00.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d897f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79d4070850 0x7f79d4148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:00.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d897f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79d4149010 0x7f79d414fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:00.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d897f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79d41494a0 con 0x7f79d40b58e0 2024-03-16T10:11:00.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d897f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f79d413fb40 con 0x7f79d4070850 2024-03-16T10:11:00.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d897f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f79d40b5cc0 con 0x7f79d4149010 2024-03-16T10:11:00.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d1d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79d4070850 0x7f79d4148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:00.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d1d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f79d4070850 0x7f79d4148ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33486/0 (socket says 172.21.15.17:33486) 2024-03-16T10:11:00.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d1d9b640 1 -- 172.21.15.17:0/1658929075 learned_addr learned my addr 172.21.15.17:0/1658929075 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:00.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d259c640 1 --1- 172.21.15.17:0/1658929075 >> v1:172.21.15.17:6789/0 conn(0x7f79d40b58e0 0x7f79d413f430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55946/0 (socket says 172.21.15.17:55946) 2024-03-16T10:11:00.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d2d9d640 1 --2- 172.21.15.17:0/1658929075 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79d4149010 0x7f79d414fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:00.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d1d9b640 1 -- 172.21.15.17:0/1658929075 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79d4149010 msgr2=0x7f79d414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:00.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d1d9b640 1 --2- 172.21.15.17:0/1658929075 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79d4149010 0x7f79d414fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:00.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d1d9b640 1 -- 172.21.15.17:0/1658929075 >> v1:172.21.15.17:6789/0 conn(0x7f79d40b58e0 legacy=0x7f79d413f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:00.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d1d9b640 1 -- 172.21.15.17:0/1658929075 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d406e830 con 0x7f79d4070850 2024-03-16T10:11:00.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.293+0000 7f79d2d9d640 1 --2- 172.21.15.17:0/1658929075 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79d4149010 0x7f79d414fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:00.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.297+0000 7f79d1d9b640 1 --2- 172.21.15.17:0/1658929075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79d4070850 0x7f79d4148ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009870 tx=0x7f79c801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74984d7ba3e09b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:00.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.297+0000 7f79d159a640 1 -- 172.21.15.17:0/1658929075 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c80b0070 con 0x7f79d4070850 2024-03-16T10:11:00.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.297+0000 7f79d159a640 1 -- 172.21.15.17:0/1658929075 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c80ab600 con 0x7f79d4070850 2024-03-16T10:11:00.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.297+0000 7f79d159a640 1 -- 172.21.15.17:0/1658929075 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c80ab9d0 con 0x7f79d4070850 2024-03-16T10:11:00.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.297+0000 7f79d897f640 1 -- 172.21.15.17:0/1658929075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79d4070850 msgr2=0x7f79d4148ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:00.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.297+0000 7f79d897f640 1 --2- 172.21.15.17:0/1658929075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79d4070850 0x7f79d4148ad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79c8009870 tx=0x7f79c801c4d0 comp rx=0 tx=0).stop 2024-03-16T10:11:00.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.297+0000 7f79d897f640 1 -- 172.21.15.17:0/1658929075 shutdown_connections 2024-03-16T10:11:00.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.297+0000 7f79d897f640 1 --2- 172.21.15.17:0/1658929075 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79d4149010 0x7f79d414fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:00.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.297+0000 7f79d897f640 1 --2- 172.21.15.17:0/1658929075 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79d4070850 0x7f79d4148ad0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:00.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.297+0000 7f79d897f640 1 -- 172.21.15.17:0/1658929075 >> 172.21.15.17:0/1658929075 conn(0x7f79d40b6770 msgr2=0x7f79d40b6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:00.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.297+0000 7f79d897f640 1 -- 172.21.15.17:0/1658929075 shutdown_connections 2024-03-16T10:11:00.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.297+0000 7f79d897f640 1 -- 172.21.15.17:0/1658929075 wait complete. 2024-03-16T10:11:00.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.297+0000 7f79d897f640 1 Processor -- start 2024-03-16T10:11:00.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.297+0000 7f79d897f640 1 -- start start 2024-03-16T10:11:00.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d897f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79d4070850 0x7f79d41324d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:00.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d897f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79d40b58e0 0x7f79d41e1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:00.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d897f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79d4149010 0x7f79d40b7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:00.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d897f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f79d4152b50 con 0x7f79d40b58e0 2024-03-16T10:11:00.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d897f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79d41529d0 con 0x7f79d4070850 2024-03-16T10:11:00.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d897f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f79d4152cd0 con 0x7f79d4149010 2024-03-16T10:11:00.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d1d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79d40b58e0 0x7f79d41e1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:00.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d1d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79d40b58e0 0x7f79d41e1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33496/0 (socket says 172.21.15.17:33496) 2024-03-16T10:11:00.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d1d9b640 1 -- 172.21.15.17:0/3608742136 learned_addr learned my addr 172.21.15.17:0/3608742136 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:00.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d259c640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79d4070850 0x7f79d41324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:00.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d2d9d640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79d4149010 0x7f79d40b7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:00.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d1d9b640 1 -- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79d4149010 msgr2=0x7f79d40b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:00.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d1d9b640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79d4149010 0x7f79d40b7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:00.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d1d9b640 1 -- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79d4070850 msgr2=0x7f79d41324d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:00.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d1d9b640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79d4070850 0x7f79d41324d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:00.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d1d9b640 1 -- 172.21.15.17:0/3608742136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d41357a0 con 0x7f79d40b58e0 2024-03-16T10:11:00.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d259c640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79d4070850 0x7f79d41324d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:00.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d1d9b640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79d40b58e0 0x7f79d41e1440 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f79c80ae040 tx=0x7f79c8019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:00.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79c2ffd640 1 -- 172.21.15.17:0/3608742136 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c80bb070 con 0x7f79d40b58e0 2024-03-16T10:11:00.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79c2ffd640 1 -- 172.21.15.17:0/3608742136 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c80b6630 con 0x7f79d40b58e0 2024-03-16T10:11:00.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79c2ffd640 1 -- 172.21.15.17:0/3608742136 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c80b0040 con 0x7f79d40b58e0 2024-03-16T10:11:00.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d897f640 1 -- 172.21.15.17:0/3608742136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d4135a40 con 0x7f79d40b58e0 2024-03-16T10:11:00.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d897f640 1 -- 172.21.15.17:0/3608742136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d40b8e70 con 0x7f79d40b58e0 2024-03-16T10:11:00.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d897f640 1 -- 172.21.15.17:0/3608742136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d406a130 con 0x7f79d40b58e0 2024-03-16T10:11:00.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79c2ffd640 1 -- 172.21.15.17:0/3608742136 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f79c80ab4c0 con 0x7f79d40b58e0 2024-03-16T10:11:00.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79c2ffd640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f79b4067660 0x7f79b4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:00.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79c2ffd640 1 -- 172.21.15.17:0/3608742136 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f79c811f110 con 0x7f79d40b58e0 2024-03-16T10:11:00.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.301+0000 7f79d259c640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f79b4067660 0x7f79b4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:00.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.305+0000 7f79d259c640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f79b4067660 0x7f79b4069b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f79c4007fd0 tx=0x7f79c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:00.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.305+0000 7f79c2ffd640 1 -- 172.21.15.17:0/3608742136 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f79c8016150 con 0x7f79d40b58e0 2024-03-16T10:11:00.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:00.505+0000 7f79d897f640 1 -- 172.21.15.17:0/3608742136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f79d40730d0 con 0x7f79d40b58e0 2024-03-16T10:11:01.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.149+0000 7f79c2ffd640 1 -- 172.21.15.17:0/3608742136 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f79c80efe20 con 0x7f79d40b58e0 2024-03-16T10:11:01.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.153+0000 7f79d897f640 1 -- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f79b4067660 msgr2=0x7f79b4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:01.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.153+0000 7f79d897f640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f79b4067660 0x7f79b4069b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f79c4007fd0 tx=0x7f79c4009040 comp rx=0 tx=0).stop 2024-03-16T10:11:01.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.153+0000 7f79d897f640 1 -- 172.21.15.17:0/3608742136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79d40b58e0 msgr2=0x7f79d41e1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:01.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.153+0000 7f79d897f640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79d40b58e0 0x7f79d41e1440 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f79c80ae040 tx=0x7f79c8019270 comp rx=0 tx=0).stop 2024-03-16T10:11:01.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.153+0000 7f79d897f640 1 -- 172.21.15.17:0/3608742136 shutdown_connections 2024-03-16T10:11:01.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.153+0000 7f79d897f640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f79b4067660 0x7f79b4069b20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.153+0000 7f79d897f640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f79d4149010 0x7f79d40b7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.153+0000 7f79d897f640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79d40b58e0 0x7f79d41e1440 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.153+0000 7f79d897f640 1 --2- 172.21.15.17:0/3608742136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79d4070850 0x7f79d41324d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.153+0000 7f79d897f640 1 -- 172.21.15.17:0/3608742136 >> 172.21.15.17:0/3608742136 conn(0x7f79d40b6770 msgr2=0x7f79d413a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:01.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.153+0000 7f79d897f640 1 -- 172.21.15.17:0/3608742136 shutdown_connections 2024-03-16T10:11:01.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.153+0000 7f79d897f640 1 -- 172.21.15.17:0/3608742136 wait complete. 2024-03-16T10:11:01.174 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-16T10:11:01.175 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:11:01.213 INFO:tasks.ceph.mds.a.smithi017.stderr:2024-03-16T10:11:01.209+0000 7faf48b4c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-16T10:11:01.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.305+0000 7f1e6014e640 1 Processor -- start 2024-03-16T10:11:01.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.305+0000 7f1e6014e640 1 -- start start 2024-03-16T10:11:01.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.305+0000 7f1e6014e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1e580b49d0 0x7f1e58150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:01.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.305+0000 7f1e6014e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e580b6fd0 con 0x7f1e580b7770 2024-03-16T10:11:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.305+0000 7f1e6014e640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e580b64f0 con 0x7f1e58073070 2024-03-16T10:11:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.305+0000 7f1e6014e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f1e580b67d0 con 0x7f1e580b49d0 2024-03-16T10:11:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.305+0000 7f1e5deea640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1e580b7770 0x7f1e58145200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55954/0 (socket says 172.21.15.17:55954) 2024-03-16T10:11:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.305+0000 7f1e5deea640 1 -- 172.21.15.17:0/869846445 learned_addr learned my addr 172.21.15.17:0/869846445 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.305+0000 7f1e5e6eb640 1 --2- 172.21.15.17:0/869846445 >> v2:172.21.15.17:3300/0 conn(0x7f1e580b49d0 0x7f1e58150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.305+0000 7f1e5cee8640 1 -- 172.21.15.17:0/869846445 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1566993376 0 0) 0x7f1e580b6fd0 con 0x7f1e580b7770 2024-03-16T10:11:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.305+0000 7f1e5cee8640 1 -- 172.21.15.17:0/869846445 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1e40003530 con 0x7f1e580b7770 2024-03-16T10:11:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.305+0000 7f1e5e6eb640 1 -- 172.21.15.17:0/869846445 >> v1:172.21.15.174:6789/0 conn(0x7f1e58073070 legacy=0x7f1e580b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.305+0000 7f1e5e6eb640 1 -- 172.21.15.17:0/869846445 >> v1:172.21.15.17:6789/0 conn(0x7f1e580b7770 legacy=0x7f1e58145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:01.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.305+0000 7f1e5e6eb640 1 -- 172.21.15.17:0/869846445 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e580b6ab0 con 0x7f1e580b49d0 2024-03-16T10:11:01.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5e6eb640 1 --2- 172.21.15.17:0/869846445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e580b49d0 0x7f1e58150ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1e500097c0 tx=0x7f1e5001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992ef94a512797b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:01.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5cee8640 1 -- 172.21.15.17:0/869846445 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e500b0070 con 0x7f1e580b49d0 2024-03-16T10:11:01.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5cee8640 1 -- 172.21.15.17:0/869846445 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e500ab600 con 0x7f1e580b49d0 2024-03-16T10:11:01.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5cee8640 1 -- 172.21.15.17:0/869846445 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e500ab9d0 con 0x7f1e580b49d0 2024-03-16T10:11:01.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 -- 172.21.15.17:0/869846445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e580b49d0 msgr2=0x7f1e58150ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:01.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 --2- 172.21.15.17:0/869846445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e580b49d0 0x7f1e58150ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1e500097c0 tx=0x7f1e5001c380 comp rx=0 tx=0).stop 2024-03-16T10:11:01.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 -- 172.21.15.17:0/869846445 shutdown_connections 2024-03-16T10:11:01.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 --2- 172.21.15.17:0/869846445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e580b49d0 0x7f1e58150ab0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 -- 172.21.15.17:0/869846445 >> 172.21.15.17:0/869846445 conn(0x7f1e5806b290 msgr2=0x7f1e5806b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:01.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 -- 172.21.15.17:0/869846445 shutdown_connections 2024-03-16T10:11:01.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 -- 172.21.15.17:0/869846445 wait complete. 2024-03-16T10:11:01.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 Processor -- start 2024-03-16T10:11:01.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 -- start start 2024-03-16T10:11:01.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e58073070 0x7f1e5813bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:01.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1e580b49d0 0x7f1e5813d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e580b7770 0x7f1e580b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1e58153990 con 0x7f1e580b7770 2024-03-16T10:11:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e58153810 con 0x7f1e58073070 2024-03-16T10:11:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1e58153b10 con 0x7f1e580b49d0 2024-03-16T10:11:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5e6eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e580b7770 0x7f1e580b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5e6eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e580b7770 0x7f1e580b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33520/0 (socket says 172.21.15.17:33520) 2024-03-16T10:11:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5e6eb640 1 -- 172.21.15.17:0/1461451237 learned_addr learned my addr 172.21.15.17:0/1461451237 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5deea640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e58073070 0x7f1e5813bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5d6e9640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1e580b49d0 0x7f1e5813d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5e6eb640 1 -- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1e580b49d0 msgr2=0x7f1e5813d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5e6eb640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1e580b49d0 0x7f1e5813d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5e6eb640 1 -- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e58073070 msgr2=0x7f1e5813bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5e6eb640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e58073070 0x7f1e5813bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5e6eb640 1 -- 172.21.15.17:0/1461451237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5813c410 con 0x7f1e580b7770 2024-03-16T10:11:01.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5d6e9640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1e580b49d0 0x7f1e5813d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:01.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e5e6eb640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e580b7770 0x7f1e580b0870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1e50018f90 tx=0x7f1e50019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:01.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e567fc640 1 -- 172.21.15.17:0/1461451237 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e500abe60 con 0x7f1e580b7770 2024-03-16T10:11:01.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e567fc640 1 -- 172.21.15.17:0/1461451237 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e500b9070 con 0x7f1e580b7770 2024-03-16T10:11:01.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 -- 172.21.15.17:0/1461451237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5813c6b0 con 0x7f1e580b7770 2024-03-16T10:11:01.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e567fc640 1 -- 172.21.15.17:0/1461451237 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e500b0040 con 0x7f1e580b7770 2024-03-16T10:11:01.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.309+0000 7f1e6014e640 1 -- 172.21.15.17:0/1461451237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5813ea10 con 0x7f1e580b7770 2024-03-16T10:11:01.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.313+0000 7f1e6014e640 1 -- 172.21.15.17:0/1461451237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e5806a130 con 0x7f1e580b7770 2024-03-16T10:11:01.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.313+0000 7f1e567fc640 1 -- 172.21.15.17:0/1461451237 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f1e500b4650 con 0x7f1e580b7770 2024-03-16T10:11:01.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.313+0000 7f1e567fc640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1e44067690 0x7f1e44069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:01.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.313+0000 7f1e567fc640 1 -- 172.21.15.17:0/1461451237 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1e5011fde0 con 0x7f1e580b7770 2024-03-16T10:11:01.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.313+0000 7f1e5deea640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1e44067690 0x7f1e44069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:01.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.313+0000 7f1e5deea640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1e44067690 0x7f1e44069b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e48001570 tx=0x7f1e48009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:01.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.317+0000 7f1e567fc640 1 -- 172.21.15.17:0/1461451237 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1e50016090 con 0x7f1e580b7770 2024-03-16T10:11:01.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.509+0000 7f1e6014e640 1 -- 172.21.15.17:0/1461451237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e58142c10 con 0x7f1e580b7770 2024-03-16T10:11:01.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.509+0000 7f1e567fc640 1 -- 172.21.15.17:0/1461451237 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7f1e500f0b50 con 0x7f1e580b7770 2024-03-16T10:11:01.512 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:01.512 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1058712158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1058712158},{"type":"v1","addr":"172.21.15.174:6835","nonce":1058712158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:10:57.464714+0000","modified":"2024-03-16T10:11:00.509595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6833/3105182870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3105182870},{"type":"v1","addr":"172.21.15.17:6833","nonce":3105182870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-16T10:11:01.513 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 8 2024-03-16T10:11:01.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.509+0000 7f1e6014e640 1 -- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1e44067690 msgr2=0x7f1e44069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:01.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.509+0000 7f1e6014e640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1e44067690 0x7f1e44069b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e48001570 tx=0x7f1e48009040 comp rx=0 tx=0).stop 2024-03-16T10:11:01.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.509+0000 7f1e6014e640 1 -- 172.21.15.17:0/1461451237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e580b7770 msgr2=0x7f1e580b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:01.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.509+0000 7f1e6014e640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e580b7770 0x7f1e580b0870 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1e50018f90 tx=0x7f1e50019120 comp rx=0 tx=0).stop 2024-03-16T10:11:01.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.509+0000 7f1e6014e640 1 -- 172.21.15.17:0/1461451237 shutdown_connections 2024-03-16T10:11:01.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.509+0000 7f1e6014e640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1e44067690 0x7f1e44069b50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.509+0000 7f1e6014e640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e580b7770 0x7f1e580b0870 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.509+0000 7f1e6014e640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1e580b49d0 0x7f1e5813d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.509+0000 7f1e6014e640 1 --2- 172.21.15.17:0/1461451237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e58073070 0x7f1e5813bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.509+0000 7f1e6014e640 1 -- 172.21.15.17:0/1461451237 >> 172.21.15.17:0/1461451237 conn(0x7f1e5806b290 msgr2=0x7f1e580b90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:01.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.509+0000 7f1e6014e640 1 -- 172.21.15.17:0/1461451237 shutdown_connections 2024-03-16T10:11:01.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.509+0000 7f1e6014e640 1 -- 172.21.15.17:0/1461451237 wait complete. 2024-03-16T10:11:01.534 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f12561a4640 1 Processor -- start 2024-03-16T10:11:01.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f12561a4640 1 -- start start 2024-03-16T10:11:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f12561a4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12500b5080 0x7f12500b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f12561a4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1250145200 0x7f1250145690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f12561a4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f125013f8b0 con 0x7f125006b4f0 2024-03-16T10:11:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f12561a4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f12500b5a20 con 0x7f1250145200 2024-03-16T10:11:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f12561a4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f12500b4910 con 0x7f12500b5080 2024-03-16T10:11:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f124ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1250145200 0x7f1250145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f124f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f125006b4f0 0x7f125013f1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55966/0 (socket says 172.21.15.17:55966) 2024-03-16T10:11:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f124ffff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1250145200 0x7f1250145690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33532/0 (socket says 172.21.15.17:33532) 2024-03-16T10:11:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f124f7fe640 1 -- 172.21.15.17:0/3144140275 learned_addr learned my addr 172.21.15.17:0/3144140275 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f1246ffd640 1 --2- 172.21.15.17:0/3144140275 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12500b5080 0x7f12500b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f124ffff640 1 -- 172.21.15.17:0/3144140275 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12500b5080 msgr2=0x7f12500b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f124ffff640 1 --2- 172.21.15.17:0/3144140275 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12500b5080 0x7f12500b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f124ffff640 1 -- 172.21.15.17:0/3144140275 >> v1:172.21.15.17:6789/0 conn(0x7f125006b4f0 legacy=0x7f125013f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:01.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f124ffff640 1 -- 172.21.15.17:0/3144140275 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f125006b8d0 con 0x7f1250145200 2024-03-16T10:11:01.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f124effd640 1 -- 172.21.15.17:0/3144140275 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 333345374 0 0) 0x7f125013f8b0 con 0x7f125006b4f0 2024-03-16T10:11:01.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f124ffff640 1 --2- 172.21.15.17:0/3144140275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1250145200 0x7f1250145690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f12480097c0 tx=0x7f124801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5193f45ccad19429 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:01.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f124effd640 1 -- 172.21.15.17:0/3144140275 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12480b0070 con 0x7f1250145200 2024-03-16T10:11:01.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f124effd640 1 -- 172.21.15.17:0/3144140275 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12480ab600 con 0x7f1250145200 2024-03-16T10:11:01.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.693+0000 7f124effd640 1 -- 172.21.15.17:0/3144140275 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12480ab9d0 con 0x7f1250145200 2024-03-16T10:11:01.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 -- 172.21.15.17:0/3144140275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1250145200 msgr2=0x7f1250145690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:01.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 --2- 172.21.15.17:0/3144140275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1250145200 0x7f1250145690 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f12480097c0 tx=0x7f124801c410 comp rx=0 tx=0).stop 2024-03-16T10:11:01.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 -- 172.21.15.17:0/3144140275 shutdown_connections 2024-03-16T10:11:01.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 --2- 172.21.15.17:0/3144140275 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1250145200 0x7f1250145690 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 --2- 172.21.15.17:0/3144140275 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12500b5080 0x7f12500b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 -- 172.21.15.17:0/3144140275 >> 172.21.15.17:0/3144140275 conn(0x7f12500ba0e0 msgr2=0x7f12500ba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:01.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 -- 172.21.15.17:0/3144140275 shutdown_connections 2024-03-16T10:11:01.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 -- 172.21.15.17:0/3144140275 wait complete. 2024-03-16T10:11:01.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 Processor -- start 2024-03-16T10:11:01.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 -- start start 2024-03-16T10:11:01.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f125006b4f0 0x7f12501dd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:01.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12500b5080 0x7f1250147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:01.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1250145200 0x7f12500b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:01.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1250152630 con 0x7f125006b4f0 2024-03-16T10:11:01.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12501524b0 con 0x7f1250145200 2024-03-16T10:11:01.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f12561a4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f12501527b0 con 0x7f12500b5080 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f124f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f125006b4f0 0x7f12501dd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f124f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f125006b4f0 0x7f12501dd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33540/0 (socket says 172.21.15.17:33540) 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f124f7fe640 1 -- 172.21.15.17:0/4260872660 learned_addr learned my addr 172.21.15.17:0/4260872660 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f1246ffd640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12500b5080 0x7f1250147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f124ffff640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1250145200 0x7f12500b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f124f7fe640 1 -- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12500b5080 msgr2=0x7f1250147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f124f7fe640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12500b5080 0x7f1250147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f124f7fe640 1 -- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1250145200 msgr2=0x7f12500b8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f124f7fe640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1250145200 0x7f12500b8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f124f7fe640 1 -- 172.21.15.17:0/4260872660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1250148bf0 con 0x7f125006b4f0 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f124ffff640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1250145200 0x7f12500b8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.697+0000 7f124f7fe640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f125006b4f0 0x7f12501dd710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f124000c7c0 tx=0x7f124000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.701+0000 7f124cff9640 1 -- 172.21.15.17:0/4260872660 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1240018070 con 0x7f125006b4f0 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.701+0000 7f124cff9640 1 -- 172.21.15.17:0/4260872660 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f124000a640 con 0x7f125006b4f0 2024-03-16T10:11:01.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.701+0000 7f12561a4640 1 -- 172.21.15.17:0/4260872660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12501326d0 con 0x7f125006b4f0 2024-03-16T10:11:01.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.701+0000 7f124cff9640 1 -- 172.21.15.17:0/4260872660 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f124000aa30 con 0x7f125006b4f0 2024-03-16T10:11:01.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.701+0000 7f12561a4640 1 -- 172.21.15.17:0/4260872660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12500b9880 con 0x7f125006b4f0 2024-03-16T10:11:01.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.701+0000 7f12561a4640 1 -- 172.21.15.17:0/4260872660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f125006a130 con 0x7f125006b4f0 2024-03-16T10:11:01.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.701+0000 7f124cff9640 1 -- 172.21.15.17:0/4260872660 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f124001b070 con 0x7f125006b4f0 2024-03-16T10:11:01.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.701+0000 7f124cff9640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1230067660 0x7f1230069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:01.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.701+0000 7f124cff9640 1 -- 172.21.15.17:0/4260872660 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1240084e10 con 0x7f125006b4f0 2024-03-16T10:11:01.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.701+0000 7f1246ffd640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1230067660 0x7f1230069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:01.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.701+0000 7f1246ffd640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1230067660 0x7f1230069b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1250073370 tx=0x7f1238008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:01.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.705+0000 7f124cff9640 1 -- 172.21.15.17:0/4260872660 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1250148bf0 con 0x7f125006b4f0 2024-03-16T10:11:01.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.917+0000 7f12561a4640 1 -- 172.21.15.17:0/4260872660 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f12500b63a0 con 0x7f125006b4f0 2024-03-16T10:11:01.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.917+0000 7f124cff9640 1 -- 172.21.15.17:0/4260872660 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7f1240055b20 con 0x7f125006b4f0 2024-03-16T10:11:01.921 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:01.921 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:01.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.921+0000 7f12561a4640 1 -- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1230067660 msgr2=0x7f1230069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:01.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.921+0000 7f12561a4640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1230067660 0x7f1230069b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1250073370 tx=0x7f1238008040 comp rx=0 tx=0).stop 2024-03-16T10:11:01.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.921+0000 7f12561a4640 1 -- 172.21.15.17:0/4260872660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f125006b4f0 msgr2=0x7f12501dd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:01.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.921+0000 7f12561a4640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f125006b4f0 0x7f12501dd710 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f124000c7c0 tx=0x7f124000cc80 comp rx=0 tx=0).stop 2024-03-16T10:11:01.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.921+0000 7f12561a4640 1 -- 172.21.15.17:0/4260872660 shutdown_connections 2024-03-16T10:11:01.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.921+0000 7f12561a4640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1230067660 0x7f1230069b20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.921+0000 7f12561a4640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1250145200 0x7f12500b8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.921+0000 7f12561a4640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f12500b5080 0x7f1250147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.921+0000 7f12561a4640 1 --2- 172.21.15.17:0/4260872660 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f125006b4f0 0x7f12501dd710 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.921+0000 7f12561a4640 1 -- 172.21.15.17:0/4260872660 >> 172.21.15.17:0/4260872660 conn(0x7f12500ba0e0 msgr2=0x7f125006db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.921+0000 7f12561a4640 1 -- 172.21.15.17:0/4260872660 shutdown_connections 2024-03-16T10:11:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:01.921+0000 7f12561a4640 1 -- 172.21.15.17:0/4260872660 wait complete. 2024-03-16T10:11:01.939 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-16T10:11:01.939 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:11:02.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.049+0000 7f0cffe0e640 1 Processor -- start 2024-03-16T10:11:02.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.049+0000 7f0cffe0e640 1 -- start start 2024-03-16T10:11:02.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.053+0000 7f0cffe0e640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0cf80af3f0 0x7f0cf80af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.053+0000 7f0cffe0e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0cf806dcf0 0x7f0cf8144270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.053+0000 7f0cffe0e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0cf80aff10 con 0x7f0cf809b110 2024-03-16T10:11:02.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.053+0000 7f0cffe0e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0cf80b5c60 con 0x7f0cf806dcf0 2024-03-16T10:11:02.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.053+0000 7f0cffe0e640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f0cf8143a70 con 0x7f0cf80af3f0 2024-03-16T10:11:02.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.053+0000 7f0cfd3a9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0cf806dcf0 0x7f0cf8144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.053+0000 7f0cfe3ab640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f0cf809b110 0x7f0cf8146ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55980/0 (socket says 172.21.15.17:55980) 2024-03-16T10:11:02.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.053+0000 7f0cfe3ab640 1 -- 172.21.15.17:0/491981961 learned_addr learned my addr 172.21.15.17:0/491981961 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:02.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.053+0000 7f0cfdbaa640 1 --2- 172.21.15.17:0/491981961 >> v2:172.21.15.174:3300/0 conn(0x7f0cf80af3f0 0x7f0cf80af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.053+0000 7f0cfd3a9640 1 -- 172.21.15.17:0/491981961 >> v2:172.21.15.174:3300/0 conn(0x7f0cf80af3f0 msgr2=0x7f0cf80af7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:02.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.053+0000 7f0cfd3a9640 1 --2- 172.21.15.17:0/491981961 >> v2:172.21.15.174:3300/0 conn(0x7f0cf80af3f0 0x7f0cf80af7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.053+0000 7f0cfd3a9640 1 -- 172.21.15.17:0/491981961 >> v1:172.21.15.17:6789/0 conn(0x7f0cf809b110 legacy=0x7f0cf8146ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.053+0000 7f0cfd3a9640 1 -- 172.21.15.17:0/491981961 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf8143d50 con 0x7f0cf806dcf0 2024-03-16T10:11:02.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfd3a9640 1 --2- 172.21.15.17:0/491981961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cf806dcf0 0x7f0cf8144270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0cf806f8e0 tx=0x7f0ce801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b47580804b6ff4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:02.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfcba8640 1 -- 172.21.15.17:0/491981961 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80b0070 con 0x7f0cf806dcf0 2024-03-16T10:11:02.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfcba8640 1 -- 172.21.15.17:0/491981961 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce80ab400 con 0x7f0cf806dcf0 2024-03-16T10:11:02.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfcba8640 1 -- 172.21.15.17:0/491981961 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80ab7d0 con 0x7f0cf806dcf0 2024-03-16T10:11:02.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 -- 172.21.15.17:0/491981961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cf806dcf0 msgr2=0x7f0cf8144270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 --2- 172.21.15.17:0/491981961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cf806dcf0 0x7f0cf8144270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0cf806f8e0 tx=0x7f0ce801c3f0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 -- 172.21.15.17:0/491981961 shutdown_connections 2024-03-16T10:11:02.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 --2- 172.21.15.17:0/491981961 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cf806dcf0 0x7f0cf8144270 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 --2- 172.21.15.17:0/491981961 >> v2:172.21.15.174:3300/0 conn(0x7f0cf80af3f0 0x7f0cf80af7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 -- 172.21.15.17:0/491981961 >> 172.21.15.17:0/491981961 conn(0x7f0cf80b5e10 msgr2=0x7f0cf80b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:02.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 -- 172.21.15.17:0/491981961 shutdown_connections 2024-03-16T10:11:02.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 -- 172.21.15.17:0/491981961 wait complete. 2024-03-16T10:11:02.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 Processor -- start 2024-03-16T10:11:02.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 -- start start 2024-03-16T10:11:02.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0cf806dcf0 0x7f0cf813f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cf809b110 0x7f0cf8137620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf80af3f0 0x7f0cf813d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf809abc0 con 0x7f0cf809b110 2024-03-16T10:11:02.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf809aa40 con 0x7f0cf80af3f0 2024-03-16T10:11:02.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0cf81475e0 con 0x7f0cf806dcf0 2024-03-16T10:11:02.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfd3a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cf809b110 0x7f0cf8137620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfd3a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cf809b110 0x7f0cf8137620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33572/0 (socket says 172.21.15.17:33572) 2024-03-16T10:11:02.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfd3a9640 1 -- 172.21.15.17:0/3483040964 learned_addr learned my addr 172.21.15.17:0/3483040964 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:02.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfe3ab640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf80af3f0 0x7f0cf813d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfdbaa640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0cf806dcf0 0x7f0cf813f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfd3a9640 1 -- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0cf806dcf0 msgr2=0x7f0cf813f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfd3a9640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0cf806dcf0 0x7f0cf813f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfd3a9640 1 -- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf80af3f0 msgr2=0x7f0cf813d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfd3a9640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf80af3f0 0x7f0cf813d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfd3a9640 1 -- 172.21.15.17:0/3483040964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf8140030 con 0x7f0cf809b110 2024-03-16T10:11:02.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfe3ab640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf80af3f0 0x7f0cf813d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:02.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cfd3a9640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cf809b110 0x7f0cf8137620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80ae040 tx=0x7f0ce8019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:02.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cee7fc640 1 -- 172.21.15.17:0/3483040964 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80b0040 con 0x7f0cf809b110 2024-03-16T10:11:02.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cee7fc640 1 -- 172.21.15.17:0/3483040964 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ce80b9070 con 0x7f0cf809b110 2024-03-16T10:11:02.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 -- 172.21.15.17:0/3483040964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf8140de0 con 0x7f0cf809b110 2024-03-16T10:11:02.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cee7fc640 1 -- 172.21.15.17:0/3483040964 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80b4690 con 0x7f0cf809b110 2024-03-16T10:11:02.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.057+0000 7f0cffe0e640 1 -- 172.21.15.17:0/3483040964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf809f3f0 con 0x7f0cf809b110 2024-03-16T10:11:02.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.061+0000 7f0cffe0e640 1 -- 172.21.15.17:0/3483040964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf806a130 con 0x7f0cf809b110 2024-03-16T10:11:02.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.061+0000 7f0cee7fc640 1 -- 172.21.15.17:0/3483040964 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f0ce80b4830 con 0x7f0cf809b110 2024-03-16T10:11:02.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.061+0000 7f0cee7fc640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0cdc067660 0x7f0cdc069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.061+0000 7f0cee7fc640 1 -- 172.21.15.17:0/3483040964 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0ce81201b0 con 0x7f0cf809b110 2024-03-16T10:11:02.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.061+0000 7f0cfdbaa640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0cdc067660 0x7f0cdc069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.061+0000 7f0cfdbaa640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0cdc067660 0x7f0cdc069b20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40096f0 tx=0x7f0cf4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:02.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.069+0000 7f0cee7fc640 1 -- 172.21.15.17:0/3483040964 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0ce8016070 con 0x7f0cf809b110 2024-03-16T10:11:02.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.225+0000 7f0cffe0e640 1 -- 172.21.15.17:0/3483040964 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0cf80b6990 con 0x7f0cdc067660 2024-03-16T10:11:02.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.241+0000 7f0cee7fc640 1 -- 172.21.15.17:0/3483040964 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251952 (secure 0 0 0) 0x7f0cf80b6990 con 0x7f0cdc067660 2024-03-16T10:11:02.246 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:02.248 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-16T10:11:02.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.245+0000 7f0cffe0e640 1 -- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0cdc067660 msgr2=0x7f0cdc069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.245+0000 7f0cffe0e640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0cdc067660 0x7f0cdc069b20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40096f0 tx=0x7f0cf4008040 comp rx=0 tx=0).stop 2024-03-16T10:11:02.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.245+0000 7f0cffe0e640 1 -- 172.21.15.17:0/3483040964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cf809b110 msgr2=0x7f0cf8137620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.245+0000 7f0cffe0e640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cf809b110 0x7f0cf8137620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ce80ae040 tx=0x7f0ce8019190 comp rx=0 tx=0).stop 2024-03-16T10:11:02.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.245+0000 7f0cffe0e640 1 -- 172.21.15.17:0/3483040964 shutdown_connections 2024-03-16T10:11:02.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.245+0000 7f0cffe0e640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0cdc067660 0x7f0cdc069b20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.245+0000 7f0cffe0e640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf80af3f0 0x7f0cf813d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.245+0000 7f0cffe0e640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cf809b110 0x7f0cf8137620 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.245+0000 7f0cffe0e640 1 --2- 172.21.15.17:0/3483040964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0cf806dcf0 0x7f0cf813f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.245+0000 7f0cffe0e640 1 -- 172.21.15.17:0/3483040964 >> 172.21.15.17:0/3483040964 conn(0x7f0cf80b5e10 msgr2=0x7f0cf80b77b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:02.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.245+0000 7f0cffe0e640 1 -- 172.21.15.17:0/3483040964 shutdown_connections 2024-03-16T10:11:02.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.245+0000 7f0cffe0e640 1 -- 172.21.15.17:0/3483040964 wait complete. 2024-03-16T10:11:02.263 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-16T10:11:02.015707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235700,"kb_used_data":2884,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739020,"statfs":{"total":773094113280,"available":772852756480,"internally_reserved":0,"allocated":2953216,"data_stored":1771496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.936136"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830585+0000","last_change":"2024-03-16T10:10:58.137685+0000","last_active":"2024-03-16T10:10:59.830585+0000","last_peered":"2024-03-16T10:10:59.830585+0000","last_clean":"2024-03-16T10:10:59.830585+0000","last_became_active":"2024-03-16T10:10:58.136911+0000","last_became_peered":"2024-03-16T10:10:58.136911+0000","last_unstale":"2024-03-16T10:10:59.830585+0000","last_undegraded":"2024-03-16T10:10:59.830585+0000","last_fullsized":"2024-03-16T10:10:59.830585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:31.763467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.117185+0000","last_change":"2024-03-16T10:10:58.131094+0000","last_active":"2024-03-16T10:10:59.117185+0000","last_peered":"2024-03-16T10:10:59.117185+0000","last_clean":"2024-03-16T10:10:59.117185+0000","last_became_active":"2024-03-16T10:10:58.130290+0000","last_became_peered":"2024-03-16T10:10:58.130290+0000","last_unstale":"2024-03-16T10:10:59.117185+0000","last_undegraded":"2024-03-16T10:10:59.117185+0000","last_fullsized":"2024-03-16T10:10:59.117185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465578+0000","last_change":"2024-03-16T10:10:58.143896+0000","last_active":"2024-03-16T10:10:59.465578+0000","last_peered":"2024-03-16T10:10:59.465578+0000","last_clean":"2024-03-16T10:10:59.465578+0000","last_became_active":"2024-03-16T10:10:58.143496+0000","last_became_peered":"2024-03-16T10:10:58.143496+0000","last_unstale":"2024-03-16T10:10:59.465578+0000","last_undegraded":"2024-03-16T10:10:59.465578+0000","last_fullsized":"2024-03-16T10:10:59.465578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:33:59.497661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123015+0000","last_change":"2024-03-16T10:10:58.150879+0000","last_active":"2024-03-16T10:10:59.123015+0000","last_peered":"2024-03-16T10:10:59.123015+0000","last_clean":"2024-03-16T10:10:59.123015+0000","last_became_active":"2024-03-16T10:10:58.149469+0000","last_became_peered":"2024-03-16T10:10:58.149469+0000","last_unstale":"2024-03-16T10:10:59.123015+0000","last_undegraded":"2024-03-16T10:10:59.123015+0000","last_fullsized":"2024-03-16T10:10:59.123015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:09:11.898626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830139+0000","last_change":"2024-03-16T10:10:58.135477+0000","last_active":"2024-03-16T10:10:59.830139+0000","last_peered":"2024-03-16T10:10:59.830139+0000","last_clean":"2024-03-16T10:10:59.830139+0000","last_became_active":"2024-03-16T10:10:58.134803+0000","last_became_peered":"2024-03-16T10:10:58.134803+0000","last_unstale":"2024-03-16T10:10:59.830139+0000","last_undegraded":"2024-03-16T10:10:59.830139+0000","last_fullsized":"2024-03-16T10:10:59.830139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:32.891269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123122+0000","last_change":"2024-03-16T10:10:58.148071+0000","last_active":"2024-03-16T10:10:59.123122+0000","last_peered":"2024-03-16T10:10:59.123122+0000","last_clean":"2024-03-16T10:10:59.123122+0000","last_became_active":"2024-03-16T10:10:58.147304+0000","last_became_peered":"2024-03-16T10:10:58.147304+0000","last_unstale":"2024-03-16T10:10:59.123122+0000","last_undegraded":"2024-03-16T10:10:59.123122+0000","last_fullsized":"2024-03-16T10:10:59.123122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:46:09.470489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829729+0000","last_change":"2024-03-16T10:10:58.129866+0000","last_active":"2024-03-16T10:10:59.829729+0000","last_peered":"2024-03-16T10:10:59.829729+0000","last_clean":"2024-03-16T10:10:59.829729+0000","last_became_active":"2024-03-16T10:10:58.129125+0000","last_became_peered":"2024-03-16T10:10:58.129125+0000","last_unstale":"2024-03-16T10:10:59.829729+0000","last_undegraded":"2024-03-16T10:10:59.829729+0000","last_fullsized":"2024-03-16T10:10:59.829729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:25:29.088835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830012+0000","last_change":"2024-03-16T10:10:58.169409+0000","last_active":"2024-03-16T10:10:59.830012+0000","last_peered":"2024-03-16T10:10:59.830012+0000","last_clean":"2024-03-16T10:10:59.830012+0000","last_became_active":"2024-03-16T10:10:58.169086+0000","last_became_peered":"2024-03-16T10:10:58.169086+0000","last_unstale":"2024-03-16T10:10:59.830012+0000","last_undegraded":"2024-03-16T10:10:59.830012+0000","last_fullsized":"2024-03-16T10:10:59.830012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830943+0000","last_change":"2024-03-16T10:10:58.175200+0000","last_active":"2024-03-16T10:10:59.830943+0000","last_peered":"2024-03-16T10:10:59.830943+0000","last_clean":"2024-03-16T10:10:59.830943+0000","last_became_active":"2024-03-16T10:10:58.174581+0000","last_became_peered":"2024-03-16T10:10:58.174581+0000","last_unstale":"2024-03-16T10:10:59.830943+0000","last_undegraded":"2024-03-16T10:10:59.830943+0000","last_fullsized":"2024-03-16T10:10:59.830943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:39.149098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120166+0000","last_change":"2024-03-16T10:10:58.141861+0000","last_active":"2024-03-16T10:10:59.120166+0000","last_peered":"2024-03-16T10:10:59.120166+0000","last_clean":"2024-03-16T10:10:59.120166+0000","last_became_active":"2024-03-16T10:10:58.139720+0000","last_became_peered":"2024-03-16T10:10:58.139720+0000","last_unstale":"2024-03-16T10:10:59.120166+0000","last_undegraded":"2024-03-16T10:10:59.120166+0000","last_fullsized":"2024-03-16T10:10:59.120166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119960+0000","last_change":"2024-03-16T10:10:58.134873+0000","last_active":"2024-03-16T10:10:59.119960+0000","last_peered":"2024-03-16T10:10:59.119960+0000","last_clean":"2024-03-16T10:10:59.119960+0000","last_became_active":"2024-03-16T10:10:58.134062+0000","last_became_peered":"2024-03-16T10:10:58.134062+0000","last_unstale":"2024-03-16T10:10:59.119960+0000","last_undegraded":"2024-03-16T10:10:59.119960+0000","last_fullsized":"2024-03-16T10:10:59.119960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121413+0000","last_change":"2024-03-16T10:10:58.132580+0000","last_active":"2024-03-16T10:10:59.121413+0000","last_peered":"2024-03-16T10:10:59.121413+0000","last_clean":"2024-03-16T10:10:59.121413+0000","last_became_active":"2024-03-16T10:10:58.130316+0000","last_became_peered":"2024-03-16T10:10:58.130316+0000","last_unstale":"2024-03-16T10:10:59.121413+0000","last_undegraded":"2024-03-16T10:10:59.121413+0000","last_fullsized":"2024-03-16T10:10:59.121413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:18.684392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.831259+0000","last_change":"2024-03-16T10:10:58.129802+0000","last_active":"2024-03-16T10:10:59.831259+0000","last_peered":"2024-03-16T10:10:59.831259+0000","last_clean":"2024-03-16T10:10:59.831259+0000","last_became_active":"2024-03-16T10:10:58.129060+0000","last_became_peered":"2024-03-16T10:10:58.129060+0000","last_unstale":"2024-03-16T10:10:59.831259+0000","last_undegraded":"2024-03-16T10:10:59.831259+0000","last_fullsized":"2024-03-16T10:10:59.831259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:09:11.898626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466226+0000","last_change":"2024-03-16T10:10:58.146456+0000","last_active":"2024-03-16T10:10:59.466226+0000","last_peered":"2024-03-16T10:10:59.466226+0000","last_clean":"2024-03-16T10:10:59.466226+0000","last_became_active":"2024-03-16T10:10:58.146121+0000","last_became_peered":"2024-03-16T10:10:58.146121+0000","last_unstale":"2024-03-16T10:10:59.466226+0000","last_undegraded":"2024-03-16T10:10:59.466226+0000","last_fullsized":"2024-03-16T10:10:59.466226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.115785+0000","last_change":"2024-03-16T10:10:58.128942+0000","last_active":"2024-03-16T10:10:59.115785+0000","last_peered":"2024-03-16T10:10:59.115785+0000","last_clean":"2024-03-16T10:10:59.115785+0000","last_became_active":"2024-03-16T10:10:58.128181+0000","last_became_peered":"2024-03-16T10:10:58.128181+0000","last_unstale":"2024-03-16T10:10:59.115785+0000","last_undegraded":"2024-03-16T10:10:59.115785+0000","last_fullsized":"2024-03-16T10:10:59.115785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122518+0000","last_change":"2024-03-16T10:10:58.149792+0000","last_active":"2024-03-16T10:10:59.122518+0000","last_peered":"2024-03-16T10:10:59.122518+0000","last_clean":"2024-03-16T10:10:59.122518+0000","last_became_active":"2024-03-16T10:10:58.148809+0000","last_became_peered":"2024-03-16T10:10:58.148809+0000","last_unstale":"2024-03-16T10:10:59.122518+0000","last_undegraded":"2024-03-16T10:10:59.122518+0000","last_fullsized":"2024-03-16T10:10:59.122518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:01:30.933303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.124057+0000","last_change":"2024-03-16T10:10:58.159388+0000","last_active":"2024-03-16T10:10:59.124057+0000","last_peered":"2024-03-16T10:10:59.124057+0000","last_clean":"2024-03-16T10:10:59.124057+0000","last_became_active":"2024-03-16T10:10:58.151577+0000","last_became_peered":"2024-03-16T10:10:58.151577+0000","last_unstale":"2024-03-16T10:10:59.124057+0000","last_undegraded":"2024-03-16T10:10:59.124057+0000","last_fullsized":"2024-03-16T10:10:59.124057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:12.960572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829702+0000","last_change":"2024-03-16T10:10:58.141141+0000","last_active":"2024-03-16T10:10:59.829702+0000","last_peered":"2024-03-16T10:10:59.829702+0000","last_clean":"2024-03-16T10:10:59.829702+0000","last_became_active":"2024-03-16T10:10:58.139690+0000","last_became_peered":"2024-03-16T10:10:58.139690+0000","last_unstale":"2024-03-16T10:10:59.829702+0000","last_undegraded":"2024-03-16T10:10:59.829702+0000","last_fullsized":"2024-03-16T10:10:59.829702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:56:48.783143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830812+0000","last_change":"2024-03-16T10:10:58.138018+0000","last_active":"2024-03-16T10:10:59.830812+0000","last_peered":"2024-03-16T10:10:59.830812+0000","last_clean":"2024-03-16T10:10:59.830812+0000","last_became_active":"2024-03-16T10:10:58.137216+0000","last_became_peered":"2024-03-16T10:10:58.137216+0000","last_unstale":"2024-03-16T10:10:59.830812+0000","last_undegraded":"2024-03-16T10:10:59.830812+0000","last_fullsized":"2024-03-16T10:10:59.830812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:08.743197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120346+0000","last_change":"2024-03-16T10:10:58.159417+0000","last_active":"2024-03-16T10:10:59.120346+0000","last_peered":"2024-03-16T10:10:59.120346+0000","last_clean":"2024-03-16T10:10:59.120346+0000","last_became_active":"2024-03-16T10:10:58.158720+0000","last_became_peered":"2024-03-16T10:10:58.158720+0000","last_unstale":"2024-03-16T10:10:59.120346+0000","last_undegraded":"2024-03-16T10:10:59.120346+0000","last_fullsized":"2024-03-16T10:10:59.120346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465780+0000","last_change":"2024-03-16T10:10:58.140446+0000","last_active":"2024-03-16T10:10:59.465780+0000","last_peered":"2024-03-16T10:10:59.465780+0000","last_clean":"2024-03-16T10:10:59.465780+0000","last_became_active":"2024-03-16T10:10:58.139239+0000","last_became_peered":"2024-03-16T10:10:58.139239+0000","last_unstale":"2024-03-16T10:10:59.465780+0000","last_undegraded":"2024-03-16T10:10:59.465780+0000","last_fullsized":"2024-03-16T10:10:59.465780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:18:51.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123714+0000","last_change":"2024-03-16T10:10:58.168915+0000","last_active":"2024-03-16T10:10:59.123714+0000","last_peered":"2024-03-16T10:10:59.123714+0000","last_clean":"2024-03-16T10:10:59.123714+0000","last_became_active":"2024-03-16T10:10:58.159348+0000","last_became_peered":"2024-03-16T10:10:58.159348+0000","last_unstale":"2024-03-16T10:10:59.123714+0000","last_undegraded":"2024-03-16T10:10:59.123714+0000","last_fullsized":"2024-03-16T10:10:59.123714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:39.149098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829952+0000","last_change":"2024-03-16T10:10:58.166358+0000","last_active":"2024-03-16T10:10:59.829952+0000","last_peered":"2024-03-16T10:10:59.829952+0000","last_clean":"2024-03-16T10:10:59.829952+0000","last_became_active":"2024-03-16T10:10:58.166047+0000","last_became_peered":"2024-03-16T10:10:58.166047+0000","last_unstale":"2024-03-16T10:10:59.829952+0000","last_undegraded":"2024-03-16T10:10:59.829952+0000","last_fullsized":"2024-03-16T10:10:59.829952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:18:51.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122656+0000","last_change":"2024-03-16T10:10:58.165962+0000","last_active":"2024-03-16T10:10:59.122656+0000","last_peered":"2024-03-16T10:10:59.122656+0000","last_clean":"2024-03-16T10:10:59.122656+0000","last_became_active":"2024-03-16T10:10:58.164489+0000","last_became_peered":"2024-03-16T10:10:58.164489+0000","last_unstale":"2024-03-16T10:10:59.122656+0000","last_undegraded":"2024-03-16T10:10:59.122656+0000","last_fullsized":"2024-03-16T10:10:59.122656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:25:29.088835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830935+0000","last_change":"2024-03-16T10:10:58.156904+0000","last_active":"2024-03-16T10:10:59.830935+0000","last_peered":"2024-03-16T10:10:59.830935+0000","last_clean":"2024-03-16T10:10:59.830935+0000","last_became_active":"2024-03-16T10:10:58.156570+0000","last_became_peered":"2024-03-16T10:10:58.156570+0000","last_unstale":"2024-03-16T10:10:59.830935+0000","last_undegraded":"2024-03-16T10:10:59.830935+0000","last_fullsized":"2024-03-16T10:10:59.830935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:01:30.933303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120153+0000","last_change":"2024-03-16T10:10:58.139427+0000","last_active":"2024-03-16T10:10:59.120153+0000","last_peered":"2024-03-16T10:10:59.120153+0000","last_clean":"2024-03-16T10:10:59.120153+0000","last_became_active":"2024-03-16T10:10:58.138492+0000","last_became_peered":"2024-03-16T10:10:58.138492+0000","last_unstale":"2024-03-16T10:10:59.120153+0000","last_undegraded":"2024-03-16T10:10:59.120153+0000","last_fullsized":"2024-03-16T10:10:59.120153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116872+0000","last_change":"2024-03-16T10:10:58.141336+0000","last_active":"2024-03-16T10:10:59.116872+0000","last_peered":"2024-03-16T10:10:59.116872+0000","last_clean":"2024-03-16T10:10:59.116872+0000","last_became_active":"2024-03-16T10:10:58.139760+0000","last_became_peered":"2024-03-16T10:10:58.139760+0000","last_unstale":"2024-03-16T10:10:59.116872+0000","last_undegraded":"2024-03-16T10:10:59.116872+0000","last_fullsized":"2024-03-16T10:10:59.116872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830701+0000","last_change":"2024-03-16T10:10:58.166524+0000","last_active":"2024-03-16T10:10:59.830701+0000","last_peered":"2024-03-16T10:10:59.830701+0000","last_clean":"2024-03-16T10:10:59.830701+0000","last_became_active":"2024-03-16T10:10:58.166197+0000","last_became_peered":"2024-03-16T10:10:58.166197+0000","last_unstale":"2024-03-16T10:10:59.830701+0000","last_undegraded":"2024-03-16T10:10:59.830701+0000","last_fullsized":"2024-03-16T10:10:59.830701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466121+0000","last_change":"2024-03-16T10:10:58.138295+0000","last_active":"2024-03-16T10:10:59.466121+0000","last_peered":"2024-03-16T10:10:59.466121+0000","last_clean":"2024-03-16T10:10:59.466121+0000","last_became_active":"2024-03-16T10:10:58.136934+0000","last_became_peered":"2024-03-16T10:10:58.136934+0000","last_unstale":"2024-03-16T10:10:59.466121+0000","last_undegraded":"2024-03-16T10:10:59.466121+0000","last_fullsized":"2024-03-16T10:10:59.466121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123600+0000","last_change":"2024-03-16T10:10:58.167036+0000","last_active":"2024-03-16T10:10:59.123600+0000","last_peered":"2024-03-16T10:10:59.123600+0000","last_clean":"2024-03-16T10:10:59.123600+0000","last_became_active":"2024-03-16T10:10:58.150783+0000","last_became_peered":"2024-03-16T10:10:58.150783+0000","last_unstale":"2024-03-16T10:10:59.123600+0000","last_undegraded":"2024-03-16T10:10:59.123600+0000","last_fullsized":"2024-03-16T10:10:59.123600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:13.329157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829447+0000","last_change":"2024-03-16T10:10:58.135886+0000","last_active":"2024-03-16T10:10:59.829447+0000","last_peered":"2024-03-16T10:10:59.829447+0000","last_clean":"2024-03-16T10:10:59.829447+0000","last_became_active":"2024-03-16T10:10:58.135152+0000","last_became_peered":"2024-03-16T10:10:58.135152+0000","last_unstale":"2024-03-16T10:10:59.829447+0000","last_undegraded":"2024-03-16T10:10:59.829447+0000","last_fullsized":"2024-03-16T10:10:59.829447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:33:04.565468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.118840+0000","last_change":"2024-03-16T10:10:58.124028+0000","last_active":"2024-03-16T10:10:59.118840+0000","last_peered":"2024-03-16T10:10:59.118840+0000","last_clean":"2024-03-16T10:10:59.118840+0000","last_became_active":"2024-03-16T10:10:58.123127+0000","last_became_peered":"2024-03-16T10:10:58.123127+0000","last_unstale":"2024-03-16T10:10:59.118840+0000","last_undegraded":"2024-03-16T10:10:59.118840+0000","last_fullsized":"2024-03-16T10:10:59.118840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:18:51.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119320+0000","last_change":"2024-03-16T10:10:58.141873+0000","last_active":"2024-03-16T10:10:59.119320+0000","last_peered":"2024-03-16T10:10:59.119320+0000","last_clean":"2024-03-16T10:10:59.119320+0000","last_became_active":"2024-03-16T10:10:58.134577+0000","last_became_peered":"2024-03-16T10:10:58.134577+0000","last_unstale":"2024-03-16T10:10:59.119320+0000","last_undegraded":"2024-03-16T10:10:59.119320+0000","last_fullsized":"2024-03-16T10:10:59.119320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:36.702176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.831022+0000","last_change":"2024-03-16T10:10:57.116491+0000","last_active":"2024-03-16T10:10:59.831022+0000","last_peered":"2024-03-16T10:10:59.831022+0000","last_clean":"2024-03-16T10:10:59.831022+0000","last_became_active":"2024-03-16T10:10:57.116165+0000","last_became_peered":"2024-03-16T10:10:57.116165+0000","last_unstale":"2024-03-16T10:10:59.831022+0000","last_undegraded":"2024-03-16T10:10:59.831022+0000","last_fullsized":"2024-03-16T10:10:59.831022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:32:37.635494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466258+0000","last_change":"2024-03-16T10:10:58.158940+0000","last_active":"2024-03-16T10:10:59.466258+0000","last_peered":"2024-03-16T10:10:59.466258+0000","last_clean":"2024-03-16T10:10:59.466258+0000","last_became_active":"2024-03-16T10:10:58.158624+0000","last_became_peered":"2024-03-16T10:10:58.158624+0000","last_unstale":"2024-03-16T10:10:59.466258+0000","last_undegraded":"2024-03-16T10:10:59.466258+0000","last_fullsized":"2024-03-16T10:10:59.466258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120492+0000","last_change":"2024-03-16T10:10:57.103389+0000","last_active":"2024-03-16T10:10:59.120492+0000","last_peered":"2024-03-16T10:10:59.120492+0000","last_clean":"2024-03-16T10:10:59.120492+0000","last_became_active":"2024-03-16T10:10:57.100522+0000","last_became_peered":"2024-03-16T10:10:57.100522+0000","last_unstale":"2024-03-16T10:10:59.120492+0000","last_undegraded":"2024-03-16T10:10:59.120492+0000","last_fullsized":"2024-03-16T10:10:59.120492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830305+0000","last_change":"2024-03-16T10:10:57.104411+0000","last_active":"2024-03-16T10:10:59.830305+0000","last_peered":"2024-03-16T10:10:59.830305+0000","last_clean":"2024-03-16T10:10:59.830305+0000","last_became_active":"2024-03-16T10:10:57.104091+0000","last_became_peered":"2024-03-16T10:10:57.104091+0000","last_unstale":"2024-03-16T10:10:59.830305+0000","last_undegraded":"2024-03-16T10:10:59.830305+0000","last_fullsized":"2024-03-16T10:10:59.830305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:46:56.525820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829910+0000","last_change":"2024-03-16T10:10:58.133972+0000","last_active":"2024-03-16T10:10:59.829910+0000","last_peered":"2024-03-16T10:10:59.829910+0000","last_clean":"2024-03-16T10:10:59.829910+0000","last_became_active":"2024-03-16T10:10:58.131756+0000","last_became_peered":"2024-03-16T10:10:58.131756+0000","last_unstale":"2024-03-16T10:10:59.829910+0000","last_undegraded":"2024-03-16T10:10:59.829910+0000","last_fullsized":"2024-03-16T10:10:59.829910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:33:59.497661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122858+0000","last_change":"2024-03-16T10:10:57.128184+0000","last_active":"2024-03-16T10:10:59.122858+0000","last_peered":"2024-03-16T10:10:59.122858+0000","last_clean":"2024-03-16T10:10:59.122858+0000","last_became_active":"2024-03-16T10:10:57.127626+0000","last_became_peered":"2024-03-16T10:10:57.127626+0000","last_unstale":"2024-03-16T10:10:59.122858+0000","last_undegraded":"2024-03-16T10:10:59.122858+0000","last_fullsized":"2024-03-16T10:10:59.122858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.115705+0000","last_change":"2024-03-16T10:10:58.133241+0000","last_active":"2024-03-16T10:10:59.115705+0000","last_peered":"2024-03-16T10:10:59.115705+0000","last_clean":"2024-03-16T10:10:59.115705+0000","last_became_active":"2024-03-16T10:10:58.132521+0000","last_became_peered":"2024-03-16T10:10:58.132521+0000","last_unstale":"2024-03-16T10:10:59.115705+0000","last_undegraded":"2024-03-16T10:10:59.115705+0000","last_fullsized":"2024-03-16T10:10:59.115705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120090+0000","last_change":"2024-03-16T10:10:57.134408+0000","last_active":"2024-03-16T10:10:59.120090+0000","last_peered":"2024-03-16T10:10:59.120090+0000","last_clean":"2024-03-16T10:10:59.120090+0000","last_became_active":"2024-03-16T10:10:57.133909+0000","last_became_peered":"2024-03-16T10:10:57.133909+0000","last_unstale":"2024-03-16T10:10:59.120090+0000","last_undegraded":"2024-03-16T10:10:59.120090+0000","last_fullsized":"2024-03-16T10:10:59.120090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:02.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123183+0000","last_change":"2024-03-16T10:10:58.158453+0000","last_active":"2024-03-16T10:10:59.123183+0000","last_peered":"2024-03-16T10:10:59.123183+0000","last_clean":"2024-03-16T10:10:59.123183+0000","last_became_active":"2024-03-16T10:10:58.151690+0000","last_became_peered":"2024-03-16T10:10:58.151690+0000","last_unstale":"2024-03-16T10:10:59.123183+0000","last_undegraded":"2024-03-16T10:10:59.123183+0000","last_fullsized":"2024-03-16T10:10:59.123183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:08.287161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123724+0000","last_change":"2024-03-16T10:10:57.137208+0000","last_active":"2024-03-16T10:10:59.123724+0000","last_peered":"2024-03-16T10:10:59.123724+0000","last_clean":"2024-03-16T10:10:59.123724+0000","last_became_active":"2024-03-16T10:10:57.136600+0000","last_became_peered":"2024-03-16T10:10:57.136600+0000","last_unstale":"2024-03-16T10:10:59.123724+0000","last_undegraded":"2024-03-16T10:10:59.123724+0000","last_fullsized":"2024-03-16T10:10:59.123724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:02.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466338+0000","last_change":"2024-03-16T10:10:58.166872+0000","last_active":"2024-03-16T10:10:59.466338+0000","last_peered":"2024-03-16T10:10:59.466338+0000","last_clean":"2024-03-16T10:10:59.466338+0000","last_became_active":"2024-03-16T10:10:58.166556+0000","last_became_peered":"2024-03-16T10:10:58.166556+0000","last_unstale":"2024-03-16T10:10:59.466338+0000","last_undegraded":"2024-03-16T10:10:59.466338+0000","last_fullsized":"2024-03-16T10:10:59.466338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116641+0000","last_change":"2024-03-16T10:10:57.131462+0000","last_active":"2024-03-16T10:10:59.116641+0000","last_peered":"2024-03-16T10:10:59.116641+0000","last_clean":"2024-03-16T10:10:59.116641+0000","last_became_active":"2024-03-16T10:10:57.131130+0000","last_became_peered":"2024-03-16T10:10:57.131130+0000","last_unstale":"2024-03-16T10:10:59.116641+0000","last_undegraded":"2024-03-16T10:10:59.116641+0000","last_fullsized":"2024-03-16T10:10:59.116641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:02.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121634+0000","last_change":"2024-03-16T10:10:58.144791+0000","last_active":"2024-03-16T10:10:59.121634+0000","last_peered":"2024-03-16T10:10:59.121634+0000","last_clean":"2024-03-16T10:10:59.121634+0000","last_became_active":"2024-03-16T10:10:58.144004+0000","last_became_peered":"2024-03-16T10:10:58.144004+0000","last_unstale":"2024-03-16T10:10:59.121634+0000","last_undegraded":"2024-03-16T10:10:59.121634+0000","last_fullsized":"2024-03-16T10:10:59.121634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:54.596181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120325+0000","last_change":"2024-03-16T10:10:57.101847+0000","last_active":"2024-03-16T10:10:59.120325+0000","last_peered":"2024-03-16T10:10:59.120325+0000","last_clean":"2024-03-16T10:10:59.120325+0000","last_became_active":"2024-03-16T10:10:57.100973+0000","last_became_peered":"2024-03-16T10:10:57.100973+0000","last_unstale":"2024-03-16T10:10:59.120325+0000","last_undegraded":"2024-03-16T10:10:59.120325+0000","last_fullsized":"2024-03-16T10:10:59.120325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830301+0000","last_change":"2024-03-16T10:10:58.141884+0000","last_active":"2024-03-16T10:10:59.830301+0000","last_peered":"2024-03-16T10:10:59.830301+0000","last_clean":"2024-03-16T10:10:59.830301+0000","last_became_active":"2024-03-16T10:10:58.140254+0000","last_became_peered":"2024-03-16T10:10:58.140254+0000","last_unstale":"2024-03-16T10:10:59.830301+0000","last_undegraded":"2024-03-16T10:10:59.830301+0000","last_fullsized":"2024-03-16T10:10:59.830301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:08.287161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466329+0000","last_change":"2024-03-16T10:10:57.117139+0000","last_active":"2024-03-16T10:10:59.466329+0000","last_peered":"2024-03-16T10:10:59.466329+0000","last_clean":"2024-03-16T10:10:59.466329+0000","last_became_active":"2024-03-16T10:10:57.111737+0000","last_became_peered":"2024-03-16T10:10:57.111737+0000","last_unstale":"2024-03-16T10:10:59.466329+0000","last_undegraded":"2024-03-16T10:10:59.466329+0000","last_fullsized":"2024-03-16T10:10:59.466329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:46:56.525820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465172+0000","last_change":"2024-03-16T10:10:58.140649+0000","last_active":"2024-03-16T10:10:59.465172+0000","last_peered":"2024-03-16T10:10:59.465172+0000","last_clean":"2024-03-16T10:10:59.465172+0000","last_became_active":"2024-03-16T10:10:58.139713+0000","last_became_peered":"2024-03-16T10:10:58.139713+0000","last_unstale":"2024-03-16T10:10:59.465172+0000","last_undegraded":"2024-03-16T10:10:59.465172+0000","last_fullsized":"2024-03-16T10:10:59.465172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:07:30.636156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123842+0000","last_change":"2024-03-16T10:10:57.138569+0000","last_active":"2024-03-16T10:10:59.123842+0000","last_peered":"2024-03-16T10:10:59.123842+0000","last_clean":"2024-03-16T10:10:59.123842+0000","last_became_active":"2024-03-16T10:10:57.136115+0000","last_became_peered":"2024-03-16T10:10:57.136115+0000","last_unstale":"2024-03-16T10:10:59.123842+0000","last_undegraded":"2024-03-16T10:10:59.123842+0000","last_fullsized":"2024-03-16T10:10:59.123842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830245+0000","last_change":"2024-03-16T10:10:58.130172+0000","last_active":"2024-03-16T10:10:59.830245+0000","last_peered":"2024-03-16T10:10:59.830245+0000","last_clean":"2024-03-16T10:10:59.830245+0000","last_became_active":"2024-03-16T10:10:58.120580+0000","last_became_peered":"2024-03-16T10:10:58.120580+0000","last_unstale":"2024-03-16T10:10:59.830245+0000","last_undegraded":"2024-03-16T10:10:59.830245+0000","last_fullsized":"2024-03-16T10:10:59.830245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:46:09.470489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120582+0000","last_change":"2024-03-16T10:10:57.104206+0000","last_active":"2024-03-16T10:10:59.120582+0000","last_peered":"2024-03-16T10:10:59.120582+0000","last_clean":"2024-03-16T10:10:59.120582+0000","last_became_active":"2024-03-16T10:10:57.100493+0000","last_became_peered":"2024-03-16T10:10:57.100493+0000","last_unstale":"2024-03-16T10:10:59.120582+0000","last_undegraded":"2024-03-16T10:10:59.120582+0000","last_fullsized":"2024-03-16T10:10:59.120582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:32:53.097656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119275+0000","last_change":"2024-03-16T10:10:58.123895+0000","last_active":"2024-03-16T10:10:59.119275+0000","last_peered":"2024-03-16T10:10:59.119275+0000","last_clean":"2024-03-16T10:10:59.119275+0000","last_became_active":"2024-03-16T10:10:58.122933+0000","last_became_peered":"2024-03-16T10:10:58.122933+0000","last_unstale":"2024-03-16T10:10:59.119275+0000","last_undegraded":"2024-03-16T10:10:59.119275+0000","last_fullsized":"2024-03-16T10:10:59.119275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829089+0000","last_change":"2024-03-16T10:10:57.105506+0000","last_active":"2024-03-16T10:10:59.829089+0000","last_peered":"2024-03-16T10:10:59.829089+0000","last_clean":"2024-03-16T10:10:59.829089+0000","last_became_active":"2024-03-16T10:10:57.104984+0000","last_became_peered":"2024-03-16T10:10:57.104984+0000","last_unstale":"2024-03-16T10:10:59.829089+0000","last_undegraded":"2024-03-16T10:10:59.829089+0000","last_fullsized":"2024-03-16T10:10:59.829089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:34:00.287180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829799+0000","last_change":"2024-03-16T10:10:58.129371+0000","last_active":"2024-03-16T10:10:59.829799+0000","last_peered":"2024-03-16T10:10:59.829799+0000","last_clean":"2024-03-16T10:10:59.829799+0000","last_became_active":"2024-03-16T10:10:58.121147+0000","last_became_peered":"2024-03-16T10:10:58.121147+0000","last_unstale":"2024-03-16T10:10:59.829799+0000","last_undegraded":"2024-03-16T10:10:59.829799+0000","last_fullsized":"2024-03-16T10:10:59.829799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:09:11.898626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121588+0000","last_change":"2024-03-16T10:10:57.093832+0000","last_active":"2024-03-16T10:10:59.121588+0000","last_peered":"2024-03-16T10:10:59.121588+0000","last_clean":"2024-03-16T10:10:59.121588+0000","last_became_active":"2024-03-16T10:10:57.093278+0000","last_became_peered":"2024-03-16T10:10:57.093278+0000","last_unstale":"2024-03-16T10:10:59.121588+0000","last_undegraded":"2024-03-16T10:10:59.121588+0000","last_fullsized":"2024-03-16T10:10:59.121588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:35:54.941983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830807+0000","last_change":"2024-03-16T10:10:58.169395+0000","last_active":"2024-03-16T10:10:59.830807+0000","last_peered":"2024-03-16T10:10:59.830807+0000","last_clean":"2024-03-16T10:10:59.830807+0000","last_became_active":"2024-03-16T10:10:58.168702+0000","last_became_peered":"2024-03-16T10:10:58.168702+0000","last_unstale":"2024-03-16T10:10:59.830807+0000","last_undegraded":"2024-03-16T10:10:59.830807+0000","last_fullsized":"2024-03-16T10:10:59.830807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:13.329157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829866+0000","last_change":"2024-03-16T10:10:57.105292+0000","last_active":"2024-03-16T10:10:59.829866+0000","last_peered":"2024-03-16T10:10:59.829866+0000","last_clean":"2024-03-16T10:10:59.829866+0000","last_became_active":"2024-03-16T10:10:57.104688+0000","last_became_peered":"2024-03-16T10:10:57.104688+0000","last_unstale":"2024-03-16T10:10:59.829866+0000","last_undegraded":"2024-03-16T10:10:59.829866+0000","last_fullsized":"2024-03-16T10:10:59.829866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830999+0000","last_change":"2024-03-16T10:10:58.170384+0000","last_active":"2024-03-16T10:10:59.830999+0000","last_peered":"2024-03-16T10:10:59.830999+0000","last_clean":"2024-03-16T10:10:59.830999+0000","last_became_active":"2024-03-16T10:10:58.169787+0000","last_became_peered":"2024-03-16T10:10:58.169787+0000","last_unstale":"2024-03-16T10:10:59.830999+0000","last_undegraded":"2024-03-16T10:10:59.830999+0000","last_fullsized":"2024-03-16T10:10:59.830999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:12.960572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116762+0000","last_change":"2024-03-16T10:10:57.111549+0000","last_active":"2024-03-16T10:10:59.116762+0000","last_peered":"2024-03-16T10:10:59.116762+0000","last_clean":"2024-03-16T10:10:59.116762+0000","last_became_active":"2024-03-16T10:10:57.111022+0000","last_became_peered":"2024-03-16T10:10:57.111022+0000","last_unstale":"2024-03-16T10:10:59.116762+0000","last_undegraded":"2024-03-16T10:10:59.116762+0000","last_fullsized":"2024-03-16T10:10:59.116762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:33:03.557374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123471+0000","last_change":"2024-03-16T10:10:58.150154+0000","last_active":"2024-03-16T10:10:59.123471+0000","last_peered":"2024-03-16T10:10:59.123471+0000","last_clean":"2024-03-16T10:10:59.123471+0000","last_became_active":"2024-03-16T10:10:58.149355+0000","last_became_peered":"2024-03-16T10:10:58.149355+0000","last_unstale":"2024-03-16T10:10:59.123471+0000","last_undegraded":"2024-03-16T10:10:59.123471+0000","last_fullsized":"2024-03-16T10:10:59.123471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:27:42.059927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829400+0000","last_change":"2024-03-16T10:10:57.129962+0000","last_active":"2024-03-16T10:10:59.829400+0000","last_peered":"2024-03-16T10:10:59.829400+0000","last_clean":"2024-03-16T10:10:59.829400+0000","last_became_active":"2024-03-16T10:10:57.129583+0000","last_became_peered":"2024-03-16T10:10:57.129583+0000","last_unstale":"2024-03-16T10:10:59.829400+0000","last_undegraded":"2024-03-16T10:10:59.829400+0000","last_fullsized":"2024-03-16T10:10:59.829400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:32:53.097656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830586+0000","last_change":"2024-03-16T10:10:58.140201+0000","last_active":"2024-03-16T10:10:59.830586+0000","last_peered":"2024-03-16T10:10:59.830586+0000","last_clean":"2024-03-16T10:10:59.830586+0000","last_became_active":"2024-03-16T10:10:58.139272+0000","last_became_peered":"2024-03-16T10:10:58.139272+0000","last_unstale":"2024-03-16T10:10:59.830586+0000","last_undegraded":"2024-03-16T10:10:59.830586+0000","last_fullsized":"2024-03-16T10:10:59.830586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:27:42.059927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829599+0000","last_change":"2024-03-16T10:10:48.248686+0000","last_active":"2024-03-16T10:10:59.829599+0000","last_peered":"2024-03-16T10:10:59.829599+0000","last_clean":"2024-03-16T10:10:59.829599+0000","last_became_active":"2024-03-16T10:10:48.243054+0000","last_became_peered":"2024-03-16T10:10:48.243054+0000","last_unstale":"2024-03-16T10:10:59.829599+0000","last_undegraded":"2024-03-16T10:10:59.829599+0000","last_fullsized":"2024-03-16T10:10:59.829599+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_clean_scrub_stamp":"2024-03-16T10:10:46.517938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:42.906473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122484+0000","last_change":"2024-03-16T10:10:57.123078+0000","last_active":"2024-03-16T10:10:59.122484+0000","last_peered":"2024-03-16T10:10:59.122484+0000","last_clean":"2024-03-16T10:10:59.122484+0000","last_became_active":"2024-03-16T10:10:57.121087+0000","last_became_peered":"2024-03-16T10:10:57.121087+0000","last_unstale":"2024-03-16T10:10:59.122484+0000","last_undegraded":"2024-03-16T10:10:59.122484+0000","last_fullsized":"2024-03-16T10:10:59.122484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830505+0000","last_change":"2024-03-16T10:10:58.171123+0000","last_active":"2024-03-16T10:10:59.830505+0000","last_peered":"2024-03-16T10:10:59.830505+0000","last_clean":"2024-03-16T10:10:59.830505+0000","last_became_active":"2024-03-16T10:10:58.170849+0000","last_became_peered":"2024-03-16T10:10:58.170849+0000","last_unstale":"2024-03-16T10:10:59.830505+0000","last_undegraded":"2024-03-16T10:10:59.830505+0000","last_fullsized":"2024-03-16T10:10:59.830505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829543+0000","last_change":"2024-03-16T10:10:57.124354+0000","last_active":"2024-03-16T10:10:59.829543+0000","last_peered":"2024-03-16T10:10:59.829543+0000","last_clean":"2024-03-16T10:10:59.829543+0000","last_became_active":"2024-03-16T10:10:57.123636+0000","last_became_peered":"2024-03-16T10:10:57.123636+0000","last_unstale":"2024-03-16T10:10:59.829543+0000","last_undegraded":"2024-03-16T10:10:59.829543+0000","last_fullsized":"2024-03-16T10:10:59.829543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465205+0000","last_change":"2024-03-16T10:10:58.158168+0000","last_active":"2024-03-16T10:10:59.465205+0000","last_peered":"2024-03-16T10:10:59.465205+0000","last_clean":"2024-03-16T10:10:59.465205+0000","last_became_active":"2024-03-16T10:10:58.157882+0000","last_became_peered":"2024-03-16T10:10:58.157882+0000","last_unstale":"2024-03-16T10:10:59.465205+0000","last_undegraded":"2024-03-16T10:10:59.465205+0000","last_fullsized":"2024-03-16T10:10:59.465205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:10:11.295606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829675+0000","last_change":"2024-03-16T10:10:57.104381+0000","last_active":"2024-03-16T10:10:59.829675+0000","last_peered":"2024-03-16T10:10:59.829675+0000","last_clean":"2024-03-16T10:10:59.829675+0000","last_became_active":"2024-03-16T10:10:57.103673+0000","last_became_peered":"2024-03-16T10:10:57.103673+0000","last_unstale":"2024-03-16T10:10:59.829675+0000","last_undegraded":"2024-03-16T10:10:59.829675+0000","last_fullsized":"2024-03-16T10:10:59.829675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:45:53.158955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.115477+0000","last_change":"2024-03-16T10:10:58.136233+0000","last_active":"2024-03-16T10:10:59.115477+0000","last_peered":"2024-03-16T10:10:59.115477+0000","last_clean":"2024-03-16T10:10:59.115477+0000","last_became_active":"2024-03-16T10:10:58.135287+0000","last_became_peered":"2024-03-16T10:10:58.135287+0000","last_unstale":"2024-03-16T10:10:59.115477+0000","last_undegraded":"2024-03-16T10:10:59.115477+0000","last_fullsized":"2024-03-16T10:10:59.115477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.124008+0000","last_change":"2024-03-16T10:10:57.132978+0000","last_active":"2024-03-16T10:10:59.124008+0000","last_peered":"2024-03-16T10:10:59.124008+0000","last_clean":"2024-03-16T10:10:59.124008+0000","last_became_active":"2024-03-16T10:10:57.132199+0000","last_became_peered":"2024-03-16T10:10:57.132199+0000","last_unstale":"2024-03-16T10:10:59.124008+0000","last_undegraded":"2024-03-16T10:10:59.124008+0000","last_fullsized":"2024-03-16T10:10:59.124008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:58.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121640+0000","last_change":"2024-03-16T10:10:58.131949+0000","last_active":"2024-03-16T10:10:59.121640+0000","last_peered":"2024-03-16T10:10:59.121640+0000","last_clean":"2024-03-16T10:10:59.121640+0000","last_became_active":"2024-03-16T10:10:58.131234+0000","last_became_peered":"2024-03-16T10:10:58.131234+0000","last_unstale":"2024-03-16T10:10:59.121640+0000","last_undegraded":"2024-03-16T10:10:59.121640+0000","last_fullsized":"2024-03-16T10:10:59.121640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:59.100283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116717+0000","last_change":"2024-03-16T10:10:57.115504+0000","last_active":"2024-03-16T10:10:59.116717+0000","last_peered":"2024-03-16T10:10:59.116717+0000","last_clean":"2024-03-16T10:10:59.116717+0000","last_became_active":"2024-03-16T10:10:57.115273+0000","last_became_peered":"2024-03-16T10:10:57.115273+0000","last_unstale":"2024-03-16T10:10:59.116717+0000","last_undegraded":"2024-03-16T10:10:59.116717+0000","last_fullsized":"2024-03-16T10:10:59.116717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:58.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123414+0000","last_change":"2024-03-16T10:10:58.150495+0000","last_active":"2024-03-16T10:10:59.123414+0000","last_peered":"2024-03-16T10:10:59.123414+0000","last_clean":"2024-03-16T10:10:59.123414+0000","last_became_active":"2024-03-16T10:10:58.149680+0000","last_became_peered":"2024-03-16T10:10:58.149680+0000","last_unstale":"2024-03-16T10:10:59.123414+0000","last_undegraded":"2024-03-16T10:10:59.123414+0000","last_fullsized":"2024-03-16T10:10:59.123414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:31.763467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830902+0000","last_change":"2024-03-16T10:10:57.128751+0000","last_active":"2024-03-16T10:10:59.830902+0000","last_peered":"2024-03-16T10:10:59.830902+0000","last_clean":"2024-03-16T10:10:59.830902+0000","last_became_active":"2024-03-16T10:10:57.127909+0000","last_became_peered":"2024-03-16T10:10:57.127909+0000","last_unstale":"2024-03-16T10:10:59.830902+0000","last_undegraded":"2024-03-16T10:10:59.830902+0000","last_fullsized":"2024-03-16T10:10:59.830902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:09:33.897012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116753+0000","last_change":"2024-03-16T10:10:58.134860+0000","last_active":"2024-03-16T10:10:59.116753+0000","last_peered":"2024-03-16T10:10:59.116753+0000","last_clean":"2024-03-16T10:10:59.116753+0000","last_became_active":"2024-03-16T10:10:58.133470+0000","last_became_peered":"2024-03-16T10:10:58.133470+0000","last_unstale":"2024-03-16T10:10:59.116753+0000","last_undegraded":"2024-03-16T10:10:59.116753+0000","last_fullsized":"2024-03-16T10:10:59.116753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.151156+0000","last_change":"2024-03-16T10:10:57.110493+0000","last_active":"2024-03-16T10:10:59.151156+0000","last_peered":"2024-03-16T10:10:59.151156+0000","last_clean":"2024-03-16T10:10:59.151156+0000","last_became_active":"2024-03-16T10:10:57.106136+0000","last_became_peered":"2024-03-16T10:10:57.106136+0000","last_unstale":"2024-03-16T10:10:59.151156+0000","last_undegraded":"2024-03-16T10:10:59.151156+0000","last_fullsized":"2024-03-16T10:10:59.151156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.831141+0000","last_change":"2024-03-16T10:10:58.140570+0000","last_active":"2024-03-16T10:10:59.831141+0000","last_peered":"2024-03-16T10:10:59.831141+0000","last_clean":"2024-03-16T10:10:59.831141+0000","last_became_active":"2024-03-16T10:10:58.139532+0000","last_became_peered":"2024-03-16T10:10:58.139532+0000","last_unstale":"2024-03-16T10:10:59.831141+0000","last_undegraded":"2024-03-16T10:10:59.831141+0000","last_fullsized":"2024-03-16T10:10:59.831141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:25:29.088835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120378+0000","last_change":"2024-03-16T10:10:57.120255+0000","last_active":"2024-03-16T10:10:59.120378+0000","last_peered":"2024-03-16T10:10:59.120378+0000","last_clean":"2024-03-16T10:10:59.120378+0000","last_became_active":"2024-03-16T10:10:57.119498+0000","last_became_peered":"2024-03-16T10:10:57.119498+0000","last_unstale":"2024-03-16T10:10:59.120378+0000","last_undegraded":"2024-03-16T10:10:59.120378+0000","last_fullsized":"2024-03-16T10:10:59.120378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465935+0000","last_change":"2024-03-16T10:10:58.129362+0000","last_active":"2024-03-16T10:10:59.465935+0000","last_peered":"2024-03-16T10:10:59.465935+0000","last_clean":"2024-03-16T10:10:59.465935+0000","last_became_active":"2024-03-16T10:10:58.128580+0000","last_became_peered":"2024-03-16T10:10:58.128580+0000","last_unstale":"2024-03-16T10:10:59.465935+0000","last_undegraded":"2024-03-16T10:10:59.465935+0000","last_fullsized":"2024-03-16T10:10:59.465935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829443+0000","last_change":"2024-03-16T10:10:57.118219+0000","last_active":"2024-03-16T10:10:59.829443+0000","last_peered":"2024-03-16T10:10:59.829443+0000","last_clean":"2024-03-16T10:10:59.829443+0000","last_became_active":"2024-03-16T10:10:57.117436+0000","last_became_peered":"2024-03-16T10:10:57.117436+0000","last_unstale":"2024-03-16T10:10:59.829443+0000","last_undegraded":"2024-03-16T10:10:59.829443+0000","last_fullsized":"2024-03-16T10:10:59.829443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119531+0000","last_change":"2024-03-16T10:10:58.143411+0000","last_active":"2024-03-16T10:10:59.119531+0000","last_peered":"2024-03-16T10:10:59.119531+0000","last_clean":"2024-03-16T10:10:59.119531+0000","last_became_active":"2024-03-16T10:10:58.142517+0000","last_became_peered":"2024-03-16T10:10:58.142517+0000","last_unstale":"2024-03-16T10:10:59.119531+0000","last_undegraded":"2024-03-16T10:10:59.119531+0000","last_fullsized":"2024-03-16T10:10:59.119531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:03.131047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116378+0000","last_change":"2024-03-16T10:10:57.116431+0000","last_active":"2024-03-16T10:10:59.116378+0000","last_peered":"2024-03-16T10:10:59.116378+0000","last_clean":"2024-03-16T10:10:59.116378+0000","last_became_active":"2024-03-16T10:10:57.115761+0000","last_became_peered":"2024-03-16T10:10:57.115761+0000","last_unstale":"2024-03-16T10:10:59.116378+0000","last_undegraded":"2024-03-16T10:10:59.116378+0000","last_fullsized":"2024-03-16T10:10:59.116378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830186+0000","last_change":"2024-03-16T10:10:58.137343+0000","last_active":"2024-03-16T10:10:59.830186+0000","last_peered":"2024-03-16T10:10:59.830186+0000","last_clean":"2024-03-16T10:10:59.830186+0000","last_became_active":"2024-03-16T10:10:58.136517+0000","last_became_peered":"2024-03-16T10:10:58.136517+0000","last_unstale":"2024-03-16T10:10:59.830186+0000","last_undegraded":"2024-03-16T10:10:59.830186+0000","last_fullsized":"2024-03-16T10:10:59.830186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116455+0000","last_change":"2024-03-16T10:10:57.100812+0000","last_active":"2024-03-16T10:10:59.116455+0000","last_peered":"2024-03-16T10:10:59.116455+0000","last_clean":"2024-03-16T10:10:59.116455+0000","last_became_active":"2024-03-16T10:10:57.100312+0000","last_became_peered":"2024-03-16T10:10:57.100312+0000","last_unstale":"2024-03-16T10:10:59.116455+0000","last_undegraded":"2024-03-16T10:10:59.116455+0000","last_fullsized":"2024-03-16T10:10:59.116455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830809+0000","last_change":"2024-03-16T10:10:58.139588+0000","last_active":"2024-03-16T10:10:59.830809+0000","last_peered":"2024-03-16T10:10:59.830809+0000","last_clean":"2024-03-16T10:10:59.830809+0000","last_became_active":"2024-03-16T10:10:58.136603+0000","last_became_peered":"2024-03-16T10:10:58.136603+0000","last_unstale":"2024-03-16T10:10:59.830809+0000","last_undegraded":"2024-03-16T10:10:59.830809+0000","last_fullsized":"2024-03-16T10:10:59.830809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830698+0000","last_change":"2024-03-16T10:10:57.127697+0000","last_active":"2024-03-16T10:10:59.830698+0000","last_peered":"2024-03-16T10:10:59.830698+0000","last_clean":"2024-03-16T10:10:59.830698+0000","last_became_active":"2024-03-16T10:10:57.126922+0000","last_became_peered":"2024-03-16T10:10:57.126922+0000","last_unstale":"2024-03-16T10:10:59.830698+0000","last_undegraded":"2024-03-16T10:10:59.830698+0000","last_fullsized":"2024-03-16T10:10:59.830698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:56:47.775049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120088+0000","last_change":"2024-03-16T10:10:58.162962+0000","last_active":"2024-03-16T10:10:59.120088+0000","last_peered":"2024-03-16T10:10:59.120088+0000","last_clean":"2024-03-16T10:10:59.120088+0000","last_became_active":"2024-03-16T10:10:58.161601+0000","last_became_peered":"2024-03-16T10:10:58.161601+0000","last_unstale":"2024-03-16T10:10:59.120088+0000","last_undegraded":"2024-03-16T10:10:59.120088+0000","last_fullsized":"2024-03-16T10:10:59.120088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123098+0000","last_change":"2024-03-16T10:10:58.160830+0000","last_active":"2024-03-16T10:10:59.123098+0000","last_peered":"2024-03-16T10:10:59.123098+0000","last_clean":"2024-03-16T10:10:59.123098+0000","last_became_active":"2024-03-16T10:10:58.160108+0000","last_became_peered":"2024-03-16T10:10:58.160108+0000","last_unstale":"2024-03-16T10:10:59.123098+0000","last_undegraded":"2024-03-16T10:10:59.123098+0000","last_fullsized":"2024-03-16T10:10:59.123098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:32.891269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830094+0000","last_change":"2024-03-16T10:10:57.138309+0000","last_active":"2024-03-16T10:10:59.830094+0000","last_peered":"2024-03-16T10:10:59.830094+0000","last_clean":"2024-03-16T10:10:59.830094+0000","last_became_active":"2024-03-16T10:10:57.137445+0000","last_became_peered":"2024-03-16T10:10:57.137445+0000","last_unstale":"2024-03-16T10:10:59.830094+0000","last_undegraded":"2024-03-16T10:10:59.830094+0000","last_fullsized":"2024-03-16T10:10:59.830094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:58.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123572+0000","last_change":"2024-03-16T10:10:58.160844+0000","last_active":"2024-03-16T10:10:59.123572+0000","last_peered":"2024-03-16T10:10:59.123572+0000","last_clean":"2024-03-16T10:10:59.123572+0000","last_became_active":"2024-03-16T10:10:58.160163+0000","last_became_peered":"2024-03-16T10:10:58.160163+0000","last_unstale":"2024-03-16T10:10:59.123572+0000","last_undegraded":"2024-03-16T10:10:59.123572+0000","last_fullsized":"2024-03-16T10:10:59.123572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:08.743197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830486+0000","last_change":"2024-03-16T10:10:57.126626+0000","last_active":"2024-03-16T10:10:59.830486+0000","last_peered":"2024-03-16T10:10:59.830486+0000","last_clean":"2024-03-16T10:10:59.830486+0000","last_became_active":"2024-03-16T10:10:57.125624+0000","last_became_peered":"2024-03-16T10:10:57.125624+0000","last_unstale":"2024-03-16T10:10:59.830486+0000","last_undegraded":"2024-03-16T10:10:59.830486+0000","last_fullsized":"2024-03-16T10:10:59.830486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:33:03.557374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465968+0000","last_change":"2024-03-16T10:10:58.149370+0000","last_active":"2024-03-16T10:10:59.465968+0000","last_peered":"2024-03-16T10:10:59.465968+0000","last_clean":"2024-03-16T10:10:59.465968+0000","last_became_active":"2024-03-16T10:10:58.149011+0000","last_became_peered":"2024-03-16T10:10:58.149011+0000","last_unstale":"2024-03-16T10:10:59.465968+0000","last_undegraded":"2024-03-16T10:10:59.465968+0000","last_fullsized":"2024-03-16T10:10:59.465968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830859+0000","last_change":"2024-03-16T10:10:57.114529+0000","last_active":"2024-03-16T10:10:59.830859+0000","last_peered":"2024-03-16T10:10:59.830859+0000","last_clean":"2024-03-16T10:10:59.830859+0000","last_became_active":"2024-03-16T10:10:57.113935+0000","last_became_peered":"2024-03-16T10:10:57.113935+0000","last_unstale":"2024-03-16T10:10:59.830859+0000","last_undegraded":"2024-03-16T10:10:59.830859+0000","last_fullsized":"2024-03-16T10:10:59.830859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:34:00.287180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830836+0000","last_change":"2024-03-16T10:10:58.138174+0000","last_active":"2024-03-16T10:10:59.830836+0000","last_peered":"2024-03-16T10:10:59.830836+0000","last_clean":"2024-03-16T10:10:59.830836+0000","last_became_active":"2024-03-16T10:10:58.136767+0000","last_became_peered":"2024-03-16T10:10:58.136767+0000","last_unstale":"2024-03-16T10:10:59.830836+0000","last_undegraded":"2024-03-16T10:10:59.830836+0000","last_fullsized":"2024-03-16T10:10:59.830836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466402+0000","last_change":"2024-03-16T10:10:57.124377+0000","last_active":"2024-03-16T10:10:59.466402+0000","last_peered":"2024-03-16T10:10:59.466402+0000","last_clean":"2024-03-16T10:10:59.466402+0000","last_became_active":"2024-03-16T10:10:57.123699+0000","last_became_peered":"2024-03-16T10:10:57.123699+0000","last_unstale":"2024-03-16T10:10:59.466402+0000","last_undegraded":"2024-03-16T10:10:59.466402+0000","last_fullsized":"2024-03-16T10:10:59.466402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:32:37.635494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":660255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":660311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:11:02.267 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:11:02.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.417+0000 7f360ec15640 1 Processor -- start 2024-03-16T10:11:02.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.417+0000 7f360ec15640 1 -- start start 2024-03-16T10:11:02.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360ec15640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f360814ad10 0x7f360814b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360ec15640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f360806d040 0x7f3608141230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360ec15640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3608141770 0x7f3608147540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360ec15640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f360806c960 con 0x7f3608141770 2024-03-16T10:11:02.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360ec15640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f360806fcb0 con 0x7f360814ad10 2024-03-16T10:11:02.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360ec15640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f360814b0f0 con 0x7f360806d040 2024-03-16T10:11:02.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360d1b2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3608141770 0x7f3608147540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360d1b2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3608141770 0x7f3608147540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33576/0 (socket says 172.21.15.17:33576) 2024-03-16T10:11:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360d1b2640 1 -- 172.21.15.17:0/1607129840 learned_addr learned my addr 172.21.15.17:0/1607129840 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f3607fff640 1 --2- 172.21.15.17:0/1607129840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f360806d040 0x7f3608141230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360c9b1640 1 --2- 172.21.15.17:0/1607129840 >> v2:172.21.15.174:3300/0 conn(0x7f360814ad10 0x7f360814b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360d1b2640 1 -- 172.21.15.17:0/1607129840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f360806d040 msgr2=0x7f3608141230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360d1b2640 1 --2- 172.21.15.17:0/1607129840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f360806d040 0x7f3608141230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360d1b2640 1 -- 172.21.15.17:0/1607129840 >> v2:172.21.15.174:3300/0 conn(0x7f360814ad10 msgr2=0x7f360814b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360d1b2640 1 --2- 172.21.15.17:0/1607129840 >> v2:172.21.15.174:3300/0 conn(0x7f360814ad10 0x7f360814b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360d1b2640 1 -- 172.21.15.17:0/1607129840 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360806f610 con 0x7f3608141770 2024-03-16T10:11:02.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f360d1b2640 1 --2- 172.21.15.17:0/1607129840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3608141770 0x7f3608147540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3608068090 tx=0x7f360001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6716bb24ee57b9af server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:02.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f36077fe640 1 -- 172.21.15.17:0/1607129840 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36000a5670 con 0x7f3608141770 2024-03-16T10:11:02.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f36077fe640 1 -- 172.21.15.17:0/1607129840 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36000a5810 con 0x7f3608141770 2024-03-16T10:11:02.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.421+0000 7f36077fe640 1 -- 172.21.15.17:0/1607129840 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36000a5be0 con 0x7f3608141770 2024-03-16T10:11:02.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 -- 172.21.15.17:0/1607129840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3608141770 msgr2=0x7f3608147540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 --2- 172.21.15.17:0/1607129840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3608141770 0x7f3608147540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3608068090 tx=0x7f360001c9f0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 -- 172.21.15.17:0/1607129840 shutdown_connections 2024-03-16T10:11:02.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 --2- 172.21.15.17:0/1607129840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3608141770 0x7f3608147540 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 --2- 172.21.15.17:0/1607129840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f360806d040 0x7f3608141230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 --2- 172.21.15.17:0/1607129840 >> v2:172.21.15.174:3300/0 conn(0x7f360814ad10 0x7f360814b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 -- 172.21.15.17:0/1607129840 >> 172.21.15.17:0/1607129840 conn(0x7f36080718a0 msgr2=0x7f3608071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 -- 172.21.15.17:0/1607129840 shutdown_connections 2024-03-16T10:11:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 -- 172.21.15.17:0/1607129840 wait complete. 2024-03-16T10:11:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 Processor -- start 2024-03-16T10:11:02.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 -- start start 2024-03-16T10:11:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f360806d040 0x7f360809fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3608141770 0x7f36081e3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f360814ad10 0x7f3608136f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f360806ed40 con 0x7f360814ad10 2024-03-16T10:11:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36080b75a0 con 0x7f3608141770 2024-03-16T10:11:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f360806eec0 con 0x7f360806d040 2024-03-16T10:11:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360d1b2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f360814ad10 0x7f3608136f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360d1b2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f360814ad10 0x7f3608136f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33580/0 (socket says 172.21.15.17:33580) 2024-03-16T10:11:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360d1b2640 1 -- 172.21.15.17:0/110040414 learned_addr learned my addr 172.21.15.17:0/110040414 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360c9b1640 1 --2- 172.21.15.17:0/110040414 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f360806d040 0x7f360809fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f3607fff640 1 --2- 172.21.15.17:0/110040414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3608141770 0x7f36081e3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360d1b2640 1 -- 172.21.15.17:0/110040414 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f360806d040 msgr2=0x7f360809fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360d1b2640 1 --2- 172.21.15.17:0/110040414 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f360806d040 0x7f360809fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360d1b2640 1 -- 172.21.15.17:0/110040414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3608141770 msgr2=0x7f36081e3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360d1b2640 1 --2- 172.21.15.17:0/110040414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3608141770 0x7f36081e3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360d1b2640 1 -- 172.21.15.17:0/110040414 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36080a0070 con 0x7f360814ad10 2024-03-16T10:11:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360d1b2640 1 --2- 172.21.15.17:0/110040414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f360814ad10 0x7f3608136f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f36000af040 tx=0x7f3600019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f36057fa640 1 -- 172.21.15.17:0/110040414 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36000bc020 con 0x7f360814ad10 2024-03-16T10:11:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f36057fa640 1 -- 172.21.15.17:0/110040414 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36000b8670 con 0x7f360814ad10 2024-03-16T10:11:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f360ec15640 1 -- 172.21.15.17:0/110040414 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36080ba590 con 0x7f360814ad10 2024-03-16T10:11:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.425+0000 7f36057fa640 1 -- 172.21.15.17:0/110040414 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600009510 con 0x7f360814ad10 2024-03-16T10:11:02.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.429+0000 7f360ec15640 1 -- 172.21.15.17:0/110040414 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36080ba2b0 con 0x7f360814ad10 2024-03-16T10:11:02.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.429+0000 7f360ec15640 1 -- 172.21.15.17:0/110040414 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f360806a130 con 0x7f360814ad10 2024-03-16T10:11:02.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.429+0000 7f36057fa640 1 -- 172.21.15.17:0/110040414 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f36000bf020 con 0x7f360814ad10 2024-03-16T10:11:02.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.429+0000 7f36057fa640 1 --2- 172.21.15.17:0/110040414 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f35f4067660 0x7f35f4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.429+0000 7f36057fa640 1 -- 172.21.15.17:0/110040414 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f36001211c0 con 0x7f360814ad10 2024-03-16T10:11:02.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.429+0000 7f360c9b1640 1 --2- 172.21.15.17:0/110040414 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f35f4067660 0x7f35f4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.429+0000 7f360c9b1640 1 --2- 172.21.15.17:0/110040414 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f35f4067660 0x7f35f4069b20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f36080a0a10 tx=0x7f35f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:02.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.433+0000 7f36057fa640 1 -- 172.21.15.17:0/110040414 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f36000165e0 con 0x7f360814ad10 2024-03-16T10:11:02.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.585+0000 7f360ec15640 1 -- 172.21.15.17:0/110040414 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3608151bc0 con 0x7f35f4067660 2024-03-16T10:11:02.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.601+0000 7f36057fa640 1 -- 172.21.15.17:0/110040414 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251952 (secure 0 0 0) 0x7f3608151bc0 con 0x7f35f4067660 2024-03-16T10:11:02.607 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:02.608 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-16T10:11:02.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.605+0000 7f360ec15640 1 -- 172.21.15.17:0/110040414 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f35f4067660 msgr2=0x7f35f4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.605+0000 7f360ec15640 1 --2- 172.21.15.17:0/110040414 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f35f4067660 0x7f35f4069b20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f36080a0a10 tx=0x7f35f8008040 comp rx=0 tx=0).stop 2024-03-16T10:11:02.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.605+0000 7f360ec15640 1 -- 172.21.15.17:0/110040414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f360814ad10 msgr2=0x7f3608136f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.605+0000 7f360ec15640 1 --2- 172.21.15.17:0/110040414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f360814ad10 0x7f3608136f60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f36000af040 tx=0x7f3600019790 comp rx=0 tx=0).stop 2024-03-16T10:11:02.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.605+0000 7f360ec15640 1 -- 172.21.15.17:0/110040414 shutdown_connections 2024-03-16T10:11:02.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.605+0000 7f360ec15640 1 --2- 172.21.15.17:0/110040414 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f35f4067660 0x7f35f4069b20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.605+0000 7f360ec15640 1 --2- 172.21.15.17:0/110040414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f360814ad10 0x7f3608136f60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.605+0000 7f360ec15640 1 --2- 172.21.15.17:0/110040414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3608141770 0x7f36081e3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.605+0000 7f360ec15640 1 --2- 172.21.15.17:0/110040414 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f360806d040 0x7f360809fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.605+0000 7f360ec15640 1 -- 172.21.15.17:0/110040414 >> 172.21.15.17:0/110040414 conn(0x7f36080718a0 msgr2=0x7f360813d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:02.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.605+0000 7f360ec15640 1 -- 172.21.15.17:0/110040414 shutdown_connections 2024-03-16T10:11:02.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.605+0000 7f360ec15640 1 -- 172.21.15.17:0/110040414 wait complete. 2024-03-16T10:11:02.624 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-16T10:11:02.015707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235700,"kb_used_data":2884,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739020,"statfs":{"total":773094113280,"available":772852756480,"internally_reserved":0,"allocated":2953216,"data_stored":1771496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.936136"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830585+0000","last_change":"2024-03-16T10:10:58.137685+0000","last_active":"2024-03-16T10:10:59.830585+0000","last_peered":"2024-03-16T10:10:59.830585+0000","last_clean":"2024-03-16T10:10:59.830585+0000","last_became_active":"2024-03-16T10:10:58.136911+0000","last_became_peered":"2024-03-16T10:10:58.136911+0000","last_unstale":"2024-03-16T10:10:59.830585+0000","last_undegraded":"2024-03-16T10:10:59.830585+0000","last_fullsized":"2024-03-16T10:10:59.830585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:31.763467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.117185+0000","last_change":"2024-03-16T10:10:58.131094+0000","last_active":"2024-03-16T10:10:59.117185+0000","last_peered":"2024-03-16T10:10:59.117185+0000","last_clean":"2024-03-16T10:10:59.117185+0000","last_became_active":"2024-03-16T10:10:58.130290+0000","last_became_peered":"2024-03-16T10:10:58.130290+0000","last_unstale":"2024-03-16T10:10:59.117185+0000","last_undegraded":"2024-03-16T10:10:59.117185+0000","last_fullsized":"2024-03-16T10:10:59.117185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465578+0000","last_change":"2024-03-16T10:10:58.143896+0000","last_active":"2024-03-16T10:10:59.465578+0000","last_peered":"2024-03-16T10:10:59.465578+0000","last_clean":"2024-03-16T10:10:59.465578+0000","last_became_active":"2024-03-16T10:10:58.143496+0000","last_became_peered":"2024-03-16T10:10:58.143496+0000","last_unstale":"2024-03-16T10:10:59.465578+0000","last_undegraded":"2024-03-16T10:10:59.465578+0000","last_fullsized":"2024-03-16T10:10:59.465578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:33:59.497661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123015+0000","last_change":"2024-03-16T10:10:58.150879+0000","last_active":"2024-03-16T10:10:59.123015+0000","last_peered":"2024-03-16T10:10:59.123015+0000","last_clean":"2024-03-16T10:10:59.123015+0000","last_became_active":"2024-03-16T10:10:58.149469+0000","last_became_peered":"2024-03-16T10:10:58.149469+0000","last_unstale":"2024-03-16T10:10:59.123015+0000","last_undegraded":"2024-03-16T10:10:59.123015+0000","last_fullsized":"2024-03-16T10:10:59.123015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:09:11.898626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830139+0000","last_change":"2024-03-16T10:10:58.135477+0000","last_active":"2024-03-16T10:10:59.830139+0000","last_peered":"2024-03-16T10:10:59.830139+0000","last_clean":"2024-03-16T10:10:59.830139+0000","last_became_active":"2024-03-16T10:10:58.134803+0000","last_became_peered":"2024-03-16T10:10:58.134803+0000","last_unstale":"2024-03-16T10:10:59.830139+0000","last_undegraded":"2024-03-16T10:10:59.830139+0000","last_fullsized":"2024-03-16T10:10:59.830139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:32.891269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123122+0000","last_change":"2024-03-16T10:10:58.148071+0000","last_active":"2024-03-16T10:10:59.123122+0000","last_peered":"2024-03-16T10:10:59.123122+0000","last_clean":"2024-03-16T10:10:59.123122+0000","last_became_active":"2024-03-16T10:10:58.147304+0000","last_became_peered":"2024-03-16T10:10:58.147304+0000","last_unstale":"2024-03-16T10:10:59.123122+0000","last_undegraded":"2024-03-16T10:10:59.123122+0000","last_fullsized":"2024-03-16T10:10:59.123122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:46:09.470489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829729+0000","last_change":"2024-03-16T10:10:58.129866+0000","last_active":"2024-03-16T10:10:59.829729+0000","last_peered":"2024-03-16T10:10:59.829729+0000","last_clean":"2024-03-16T10:10:59.829729+0000","last_became_active":"2024-03-16T10:10:58.129125+0000","last_became_peered":"2024-03-16T10:10:58.129125+0000","last_unstale":"2024-03-16T10:10:59.829729+0000","last_undegraded":"2024-03-16T10:10:59.829729+0000","last_fullsized":"2024-03-16T10:10:59.829729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:25:29.088835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830012+0000","last_change":"2024-03-16T10:10:58.169409+0000","last_active":"2024-03-16T10:10:59.830012+0000","last_peered":"2024-03-16T10:10:59.830012+0000","last_clean":"2024-03-16T10:10:59.830012+0000","last_became_active":"2024-03-16T10:10:58.169086+0000","last_became_peered":"2024-03-16T10:10:58.169086+0000","last_unstale":"2024-03-16T10:10:59.830012+0000","last_undegraded":"2024-03-16T10:10:59.830012+0000","last_fullsized":"2024-03-16T10:10:59.830012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830943+0000","last_change":"2024-03-16T10:10:58.175200+0000","last_active":"2024-03-16T10:10:59.830943+0000","last_peered":"2024-03-16T10:10:59.830943+0000","last_clean":"2024-03-16T10:10:59.830943+0000","last_became_active":"2024-03-16T10:10:58.174581+0000","last_became_peered":"2024-03-16T10:10:58.174581+0000","last_unstale":"2024-03-16T10:10:59.830943+0000","last_undegraded":"2024-03-16T10:10:59.830943+0000","last_fullsized":"2024-03-16T10:10:59.830943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:39.149098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120166+0000","last_change":"2024-03-16T10:10:58.141861+0000","last_active":"2024-03-16T10:10:59.120166+0000","last_peered":"2024-03-16T10:10:59.120166+0000","last_clean":"2024-03-16T10:10:59.120166+0000","last_became_active":"2024-03-16T10:10:58.139720+0000","last_became_peered":"2024-03-16T10:10:58.139720+0000","last_unstale":"2024-03-16T10:10:59.120166+0000","last_undegraded":"2024-03-16T10:10:59.120166+0000","last_fullsized":"2024-03-16T10:10:59.120166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119960+0000","last_change":"2024-03-16T10:10:58.134873+0000","last_active":"2024-03-16T10:10:59.119960+0000","last_peered":"2024-03-16T10:10:59.119960+0000","last_clean":"2024-03-16T10:10:59.119960+0000","last_became_active":"2024-03-16T10:10:58.134062+0000","last_became_peered":"2024-03-16T10:10:58.134062+0000","last_unstale":"2024-03-16T10:10:59.119960+0000","last_undegraded":"2024-03-16T10:10:59.119960+0000","last_fullsized":"2024-03-16T10:10:59.119960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121413+0000","last_change":"2024-03-16T10:10:58.132580+0000","last_active":"2024-03-16T10:10:59.121413+0000","last_peered":"2024-03-16T10:10:59.121413+0000","last_clean":"2024-03-16T10:10:59.121413+0000","last_became_active":"2024-03-16T10:10:58.130316+0000","last_became_peered":"2024-03-16T10:10:58.130316+0000","last_unstale":"2024-03-16T10:10:59.121413+0000","last_undegraded":"2024-03-16T10:10:59.121413+0000","last_fullsized":"2024-03-16T10:10:59.121413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:18.684392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.831259+0000","last_change":"2024-03-16T10:10:58.129802+0000","last_active":"2024-03-16T10:10:59.831259+0000","last_peered":"2024-03-16T10:10:59.831259+0000","last_clean":"2024-03-16T10:10:59.831259+0000","last_became_active":"2024-03-16T10:10:58.129060+0000","last_became_peered":"2024-03-16T10:10:58.129060+0000","last_unstale":"2024-03-16T10:10:59.831259+0000","last_undegraded":"2024-03-16T10:10:59.831259+0000","last_fullsized":"2024-03-16T10:10:59.831259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:09:11.898626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466226+0000","last_change":"2024-03-16T10:10:58.146456+0000","last_active":"2024-03-16T10:10:59.466226+0000","last_peered":"2024-03-16T10:10:59.466226+0000","last_clean":"2024-03-16T10:10:59.466226+0000","last_became_active":"2024-03-16T10:10:58.146121+0000","last_became_peered":"2024-03-16T10:10:58.146121+0000","last_unstale":"2024-03-16T10:10:59.466226+0000","last_undegraded":"2024-03-16T10:10:59.466226+0000","last_fullsized":"2024-03-16T10:10:59.466226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.115785+0000","last_change":"2024-03-16T10:10:58.128942+0000","last_active":"2024-03-16T10:10:59.115785+0000","last_peered":"2024-03-16T10:10:59.115785+0000","last_clean":"2024-03-16T10:10:59.115785+0000","last_became_active":"2024-03-16T10:10:58.128181+0000","last_became_peered":"2024-03-16T10:10:58.128181+0000","last_unstale":"2024-03-16T10:10:59.115785+0000","last_undegraded":"2024-03-16T10:10:59.115785+0000","last_fullsized":"2024-03-16T10:10:59.115785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122518+0000","last_change":"2024-03-16T10:10:58.149792+0000","last_active":"2024-03-16T10:10:59.122518+0000","last_peered":"2024-03-16T10:10:59.122518+0000","last_clean":"2024-03-16T10:10:59.122518+0000","last_became_active":"2024-03-16T10:10:58.148809+0000","last_became_peered":"2024-03-16T10:10:58.148809+0000","last_unstale":"2024-03-16T10:10:59.122518+0000","last_undegraded":"2024-03-16T10:10:59.122518+0000","last_fullsized":"2024-03-16T10:10:59.122518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:01:30.933303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.124057+0000","last_change":"2024-03-16T10:10:58.159388+0000","last_active":"2024-03-16T10:10:59.124057+0000","last_peered":"2024-03-16T10:10:59.124057+0000","last_clean":"2024-03-16T10:10:59.124057+0000","last_became_active":"2024-03-16T10:10:58.151577+0000","last_became_peered":"2024-03-16T10:10:58.151577+0000","last_unstale":"2024-03-16T10:10:59.124057+0000","last_undegraded":"2024-03-16T10:10:59.124057+0000","last_fullsized":"2024-03-16T10:10:59.124057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:12.960572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829702+0000","last_change":"2024-03-16T10:10:58.141141+0000","last_active":"2024-03-16T10:10:59.829702+0000","last_peered":"2024-03-16T10:10:59.829702+0000","last_clean":"2024-03-16T10:10:59.829702+0000","last_became_active":"2024-03-16T10:10:58.139690+0000","last_became_peered":"2024-03-16T10:10:58.139690+0000","last_unstale":"2024-03-16T10:10:59.829702+0000","last_undegraded":"2024-03-16T10:10:59.829702+0000","last_fullsized":"2024-03-16T10:10:59.829702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:56:48.783143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830812+0000","last_change":"2024-03-16T10:10:58.138018+0000","last_active":"2024-03-16T10:10:59.830812+0000","last_peered":"2024-03-16T10:10:59.830812+0000","last_clean":"2024-03-16T10:10:59.830812+0000","last_became_active":"2024-03-16T10:10:58.137216+0000","last_became_peered":"2024-03-16T10:10:58.137216+0000","last_unstale":"2024-03-16T10:10:59.830812+0000","last_undegraded":"2024-03-16T10:10:59.830812+0000","last_fullsized":"2024-03-16T10:10:59.830812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:08.743197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120346+0000","last_change":"2024-03-16T10:10:58.159417+0000","last_active":"2024-03-16T10:10:59.120346+0000","last_peered":"2024-03-16T10:10:59.120346+0000","last_clean":"2024-03-16T10:10:59.120346+0000","last_became_active":"2024-03-16T10:10:58.158720+0000","last_became_peered":"2024-03-16T10:10:58.158720+0000","last_unstale":"2024-03-16T10:10:59.120346+0000","last_undegraded":"2024-03-16T10:10:59.120346+0000","last_fullsized":"2024-03-16T10:10:59.120346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465780+0000","last_change":"2024-03-16T10:10:58.140446+0000","last_active":"2024-03-16T10:10:59.465780+0000","last_peered":"2024-03-16T10:10:59.465780+0000","last_clean":"2024-03-16T10:10:59.465780+0000","last_became_active":"2024-03-16T10:10:58.139239+0000","last_became_peered":"2024-03-16T10:10:58.139239+0000","last_unstale":"2024-03-16T10:10:59.465780+0000","last_undegraded":"2024-03-16T10:10:59.465780+0000","last_fullsized":"2024-03-16T10:10:59.465780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:18:51.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123714+0000","last_change":"2024-03-16T10:10:58.168915+0000","last_active":"2024-03-16T10:10:59.123714+0000","last_peered":"2024-03-16T10:10:59.123714+0000","last_clean":"2024-03-16T10:10:59.123714+0000","last_became_active":"2024-03-16T10:10:58.159348+0000","last_became_peered":"2024-03-16T10:10:58.159348+0000","last_unstale":"2024-03-16T10:10:59.123714+0000","last_undegraded":"2024-03-16T10:10:59.123714+0000","last_fullsized":"2024-03-16T10:10:59.123714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:39.149098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829952+0000","last_change":"2024-03-16T10:10:58.166358+0000","last_active":"2024-03-16T10:10:59.829952+0000","last_peered":"2024-03-16T10:10:59.829952+0000","last_clean":"2024-03-16T10:10:59.829952+0000","last_became_active":"2024-03-16T10:10:58.166047+0000","last_became_peered":"2024-03-16T10:10:58.166047+0000","last_unstale":"2024-03-16T10:10:59.829952+0000","last_undegraded":"2024-03-16T10:10:59.829952+0000","last_fullsized":"2024-03-16T10:10:59.829952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:18:51.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122656+0000","last_change":"2024-03-16T10:10:58.165962+0000","last_active":"2024-03-16T10:10:59.122656+0000","last_peered":"2024-03-16T10:10:59.122656+0000","last_clean":"2024-03-16T10:10:59.122656+0000","last_became_active":"2024-03-16T10:10:58.164489+0000","last_became_peered":"2024-03-16T10:10:58.164489+0000","last_unstale":"2024-03-16T10:10:59.122656+0000","last_undegraded":"2024-03-16T10:10:59.122656+0000","last_fullsized":"2024-03-16T10:10:59.122656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:25:29.088835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830935+0000","last_change":"2024-03-16T10:10:58.156904+0000","last_active":"2024-03-16T10:10:59.830935+0000","last_peered":"2024-03-16T10:10:59.830935+0000","last_clean":"2024-03-16T10:10:59.830935+0000","last_became_active":"2024-03-16T10:10:58.156570+0000","last_became_peered":"2024-03-16T10:10:58.156570+0000","last_unstale":"2024-03-16T10:10:59.830935+0000","last_undegraded":"2024-03-16T10:10:59.830935+0000","last_fullsized":"2024-03-16T10:10:59.830935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:01:30.933303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120153+0000","last_change":"2024-03-16T10:10:58.139427+0000","last_active":"2024-03-16T10:10:59.120153+0000","last_peered":"2024-03-16T10:10:59.120153+0000","last_clean":"2024-03-16T10:10:59.120153+0000","last_became_active":"2024-03-16T10:10:58.138492+0000","last_became_peered":"2024-03-16T10:10:58.138492+0000","last_unstale":"2024-03-16T10:10:59.120153+0000","last_undegraded":"2024-03-16T10:10:59.120153+0000","last_fullsized":"2024-03-16T10:10:59.120153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116872+0000","last_change":"2024-03-16T10:10:58.141336+0000","last_active":"2024-03-16T10:10:59.116872+0000","last_peered":"2024-03-16T10:10:59.116872+0000","last_clean":"2024-03-16T10:10:59.116872+0000","last_became_active":"2024-03-16T10:10:58.139760+0000","last_became_peered":"2024-03-16T10:10:58.139760+0000","last_unstale":"2024-03-16T10:10:59.116872+0000","last_undegraded":"2024-03-16T10:10:59.116872+0000","last_fullsized":"2024-03-16T10:10:59.116872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830701+0000","last_change":"2024-03-16T10:10:58.166524+0000","last_active":"2024-03-16T10:10:59.830701+0000","last_peered":"2024-03-16T10:10:59.830701+0000","last_clean":"2024-03-16T10:10:59.830701+0000","last_became_active":"2024-03-16T10:10:58.166197+0000","last_became_peered":"2024-03-16T10:10:58.166197+0000","last_unstale":"2024-03-16T10:10:59.830701+0000","last_undegraded":"2024-03-16T10:10:59.830701+0000","last_fullsized":"2024-03-16T10:10:59.830701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466121+0000","last_change":"2024-03-16T10:10:58.138295+0000","last_active":"2024-03-16T10:10:59.466121+0000","last_peered":"2024-03-16T10:10:59.466121+0000","last_clean":"2024-03-16T10:10:59.466121+0000","last_became_active":"2024-03-16T10:10:58.136934+0000","last_became_peered":"2024-03-16T10:10:58.136934+0000","last_unstale":"2024-03-16T10:10:59.466121+0000","last_undegraded":"2024-03-16T10:10:59.466121+0000","last_fullsized":"2024-03-16T10:10:59.466121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123600+0000","last_change":"2024-03-16T10:10:58.167036+0000","last_active":"2024-03-16T10:10:59.123600+0000","last_peered":"2024-03-16T10:10:59.123600+0000","last_clean":"2024-03-16T10:10:59.123600+0000","last_became_active":"2024-03-16T10:10:58.150783+0000","last_became_peered":"2024-03-16T10:10:58.150783+0000","last_unstale":"2024-03-16T10:10:59.123600+0000","last_undegraded":"2024-03-16T10:10:59.123600+0000","last_fullsized":"2024-03-16T10:10:59.123600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:13.329157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829447+0000","last_change":"2024-03-16T10:10:58.135886+0000","last_active":"2024-03-16T10:10:59.829447+0000","last_peered":"2024-03-16T10:10:59.829447+0000","last_clean":"2024-03-16T10:10:59.829447+0000","last_became_active":"2024-03-16T10:10:58.135152+0000","last_became_peered":"2024-03-16T10:10:58.135152+0000","last_unstale":"2024-03-16T10:10:59.829447+0000","last_undegraded":"2024-03-16T10:10:59.829447+0000","last_fullsized":"2024-03-16T10:10:59.829447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:33:04.565468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.118840+0000","last_change":"2024-03-16T10:10:58.124028+0000","last_active":"2024-03-16T10:10:59.118840+0000","last_peered":"2024-03-16T10:10:59.118840+0000","last_clean":"2024-03-16T10:10:59.118840+0000","last_became_active":"2024-03-16T10:10:58.123127+0000","last_became_peered":"2024-03-16T10:10:58.123127+0000","last_unstale":"2024-03-16T10:10:59.118840+0000","last_undegraded":"2024-03-16T10:10:59.118840+0000","last_fullsized":"2024-03-16T10:10:59.118840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:18:51.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119320+0000","last_change":"2024-03-16T10:10:58.141873+0000","last_active":"2024-03-16T10:10:59.119320+0000","last_peered":"2024-03-16T10:10:59.119320+0000","last_clean":"2024-03-16T10:10:59.119320+0000","last_became_active":"2024-03-16T10:10:58.134577+0000","last_became_peered":"2024-03-16T10:10:58.134577+0000","last_unstale":"2024-03-16T10:10:59.119320+0000","last_undegraded":"2024-03-16T10:10:59.119320+0000","last_fullsized":"2024-03-16T10:10:59.119320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:36.702176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.831022+0000","last_change":"2024-03-16T10:10:57.116491+0000","last_active":"2024-03-16T10:10:59.831022+0000","last_peered":"2024-03-16T10:10:59.831022+0000","last_clean":"2024-03-16T10:10:59.831022+0000","last_became_active":"2024-03-16T10:10:57.116165+0000","last_became_peered":"2024-03-16T10:10:57.116165+0000","last_unstale":"2024-03-16T10:10:59.831022+0000","last_undegraded":"2024-03-16T10:10:59.831022+0000","last_fullsized":"2024-03-16T10:10:59.831022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:32:37.635494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466258+0000","last_change":"2024-03-16T10:10:58.158940+0000","last_active":"2024-03-16T10:10:59.466258+0000","last_peered":"2024-03-16T10:10:59.466258+0000","last_clean":"2024-03-16T10:10:59.466258+0000","last_became_active":"2024-03-16T10:10:58.158624+0000","last_became_peered":"2024-03-16T10:10:58.158624+0000","last_unstale":"2024-03-16T10:10:59.466258+0000","last_undegraded":"2024-03-16T10:10:59.466258+0000","last_fullsized":"2024-03-16T10:10:59.466258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120492+0000","last_change":"2024-03-16T10:10:57.103389+0000","last_active":"2024-03-16T10:10:59.120492+0000","last_peered":"2024-03-16T10:10:59.120492+0000","last_clean":"2024-03-16T10:10:59.120492+0000","last_became_active":"2024-03-16T10:10:57.100522+0000","last_became_peered":"2024-03-16T10:10:57.100522+0000","last_unstale":"2024-03-16T10:10:59.120492+0000","last_undegraded":"2024-03-16T10:10:59.120492+0000","last_fullsized":"2024-03-16T10:10:59.120492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830305+0000","last_change":"2024-03-16T10:10:57.104411+0000","last_active":"2024-03-16T10:10:59.830305+0000","last_peered":"2024-03-16T10:10:59.830305+0000","last_clean":"2024-03-16T10:10:59.830305+0000","last_became_active":"2024-03-16T10:10:57.104091+0000","last_became_peered":"2024-03-16T10:10:57.104091+0000","last_unstale":"2024-03-16T10:10:59.830305+0000","last_undegraded":"2024-03-16T10:10:59.830305+0000","last_fullsized":"2024-03-16T10:10:59.830305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:46:56.525820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829910+0000","last_change":"2024-03-16T10:10:58.133972+0000","last_active":"2024-03-16T10:10:59.829910+0000","last_peered":"2024-03-16T10:10:59.829910+0000","last_clean":"2024-03-16T10:10:59.829910+0000","last_became_active":"2024-03-16T10:10:58.131756+0000","last_became_peered":"2024-03-16T10:10:58.131756+0000","last_unstale":"2024-03-16T10:10:59.829910+0000","last_undegraded":"2024-03-16T10:10:59.829910+0000","last_fullsized":"2024-03-16T10:10:59.829910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:33:59.497661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122858+0000","last_change":"2024-03-16T10:10:57.128184+0000","last_active":"2024-03-16T10:10:59.122858+0000","last_peered":"2024-03-16T10:10:59.122858+0000","last_clean":"2024-03-16T10:10:59.122858+0000","last_became_active":"2024-03-16T10:10:57.127626+0000","last_became_peered":"2024-03-16T10:10:57.127626+0000","last_unstale":"2024-03-16T10:10:59.122858+0000","last_undegraded":"2024-03-16T10:10:59.122858+0000","last_fullsized":"2024-03-16T10:10:59.122858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.115705+0000","last_change":"2024-03-16T10:10:58.133241+0000","last_active":"2024-03-16T10:10:59.115705+0000","last_peered":"2024-03-16T10:10:59.115705+0000","last_clean":"2024-03-16T10:10:59.115705+0000","last_became_active":"2024-03-16T10:10:58.132521+0000","last_became_peered":"2024-03-16T10:10:58.132521+0000","last_unstale":"2024-03-16T10:10:59.115705+0000","last_undegraded":"2024-03-16T10:10:59.115705+0000","last_fullsized":"2024-03-16T10:10:59.115705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120090+0000","last_change":"2024-03-16T10:10:57.134408+0000","last_active":"2024-03-16T10:10:59.120090+0000","last_peered":"2024-03-16T10:10:59.120090+0000","last_clean":"2024-03-16T10:10:59.120090+0000","last_became_active":"2024-03-16T10:10:57.133909+0000","last_became_peered":"2024-03-16T10:10:57.133909+0000","last_unstale":"2024-03-16T10:10:59.120090+0000","last_undegraded":"2024-03-16T10:10:59.120090+0000","last_fullsized":"2024-03-16T10:10:59.120090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:02.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123183+0000","last_change":"2024-03-16T10:10:58.158453+0000","last_active":"2024-03-16T10:10:59.123183+0000","last_peered":"2024-03-16T10:10:59.123183+0000","last_clean":"2024-03-16T10:10:59.123183+0000","last_became_active":"2024-03-16T10:10:58.151690+0000","last_became_peered":"2024-03-16T10:10:58.151690+0000","last_unstale":"2024-03-16T10:10:59.123183+0000","last_undegraded":"2024-03-16T10:10:59.123183+0000","last_fullsized":"2024-03-16T10:10:59.123183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:08.287161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123724+0000","last_change":"2024-03-16T10:10:57.137208+0000","last_active":"2024-03-16T10:10:59.123724+0000","last_peered":"2024-03-16T10:10:59.123724+0000","last_clean":"2024-03-16T10:10:59.123724+0000","last_became_active":"2024-03-16T10:10:57.136600+0000","last_became_peered":"2024-03-16T10:10:57.136600+0000","last_unstale":"2024-03-16T10:10:59.123724+0000","last_undegraded":"2024-03-16T10:10:59.123724+0000","last_fullsized":"2024-03-16T10:10:59.123724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:02.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466338+0000","last_change":"2024-03-16T10:10:58.166872+0000","last_active":"2024-03-16T10:10:59.466338+0000","last_peered":"2024-03-16T10:10:59.466338+0000","last_clean":"2024-03-16T10:10:59.466338+0000","last_became_active":"2024-03-16T10:10:58.166556+0000","last_became_peered":"2024-03-16T10:10:58.166556+0000","last_unstale":"2024-03-16T10:10:59.466338+0000","last_undegraded":"2024-03-16T10:10:59.466338+0000","last_fullsized":"2024-03-16T10:10:59.466338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116641+0000","last_change":"2024-03-16T10:10:57.131462+0000","last_active":"2024-03-16T10:10:59.116641+0000","last_peered":"2024-03-16T10:10:59.116641+0000","last_clean":"2024-03-16T10:10:59.116641+0000","last_became_active":"2024-03-16T10:10:57.131130+0000","last_became_peered":"2024-03-16T10:10:57.131130+0000","last_unstale":"2024-03-16T10:10:59.116641+0000","last_undegraded":"2024-03-16T10:10:59.116641+0000","last_fullsized":"2024-03-16T10:10:59.116641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:02.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121634+0000","last_change":"2024-03-16T10:10:58.144791+0000","last_active":"2024-03-16T10:10:59.121634+0000","last_peered":"2024-03-16T10:10:59.121634+0000","last_clean":"2024-03-16T10:10:59.121634+0000","last_became_active":"2024-03-16T10:10:58.144004+0000","last_became_peered":"2024-03-16T10:10:58.144004+0000","last_unstale":"2024-03-16T10:10:59.121634+0000","last_undegraded":"2024-03-16T10:10:59.121634+0000","last_fullsized":"2024-03-16T10:10:59.121634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:54.596181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120325+0000","last_change":"2024-03-16T10:10:57.101847+0000","last_active":"2024-03-16T10:10:59.120325+0000","last_peered":"2024-03-16T10:10:59.120325+0000","last_clean":"2024-03-16T10:10:59.120325+0000","last_became_active":"2024-03-16T10:10:57.100973+0000","last_became_peered":"2024-03-16T10:10:57.100973+0000","last_unstale":"2024-03-16T10:10:59.120325+0000","last_undegraded":"2024-03-16T10:10:59.120325+0000","last_fullsized":"2024-03-16T10:10:59.120325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830301+0000","last_change":"2024-03-16T10:10:58.141884+0000","last_active":"2024-03-16T10:10:59.830301+0000","last_peered":"2024-03-16T10:10:59.830301+0000","last_clean":"2024-03-16T10:10:59.830301+0000","last_became_active":"2024-03-16T10:10:58.140254+0000","last_became_peered":"2024-03-16T10:10:58.140254+0000","last_unstale":"2024-03-16T10:10:59.830301+0000","last_undegraded":"2024-03-16T10:10:59.830301+0000","last_fullsized":"2024-03-16T10:10:59.830301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:08.287161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466329+0000","last_change":"2024-03-16T10:10:57.117139+0000","last_active":"2024-03-16T10:10:59.466329+0000","last_peered":"2024-03-16T10:10:59.466329+0000","last_clean":"2024-03-16T10:10:59.466329+0000","last_became_active":"2024-03-16T10:10:57.111737+0000","last_became_peered":"2024-03-16T10:10:57.111737+0000","last_unstale":"2024-03-16T10:10:59.466329+0000","last_undegraded":"2024-03-16T10:10:59.466329+0000","last_fullsized":"2024-03-16T10:10:59.466329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:46:56.525820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465172+0000","last_change":"2024-03-16T10:10:58.140649+0000","last_active":"2024-03-16T10:10:59.465172+0000","last_peered":"2024-03-16T10:10:59.465172+0000","last_clean":"2024-03-16T10:10:59.465172+0000","last_became_active":"2024-03-16T10:10:58.139713+0000","last_became_peered":"2024-03-16T10:10:58.139713+0000","last_unstale":"2024-03-16T10:10:59.465172+0000","last_undegraded":"2024-03-16T10:10:59.465172+0000","last_fullsized":"2024-03-16T10:10:59.465172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:07:30.636156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123842+0000","last_change":"2024-03-16T10:10:57.138569+0000","last_active":"2024-03-16T10:10:59.123842+0000","last_peered":"2024-03-16T10:10:59.123842+0000","last_clean":"2024-03-16T10:10:59.123842+0000","last_became_active":"2024-03-16T10:10:57.136115+0000","last_became_peered":"2024-03-16T10:10:57.136115+0000","last_unstale":"2024-03-16T10:10:59.123842+0000","last_undegraded":"2024-03-16T10:10:59.123842+0000","last_fullsized":"2024-03-16T10:10:59.123842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830245+0000","last_change":"2024-03-16T10:10:58.130172+0000","last_active":"2024-03-16T10:10:59.830245+0000","last_peered":"2024-03-16T10:10:59.830245+0000","last_clean":"2024-03-16T10:10:59.830245+0000","last_became_active":"2024-03-16T10:10:58.120580+0000","last_became_peered":"2024-03-16T10:10:58.120580+0000","last_unstale":"2024-03-16T10:10:59.830245+0000","last_undegraded":"2024-03-16T10:10:59.830245+0000","last_fullsized":"2024-03-16T10:10:59.830245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:46:09.470489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120582+0000","last_change":"2024-03-16T10:10:57.104206+0000","last_active":"2024-03-16T10:10:59.120582+0000","last_peered":"2024-03-16T10:10:59.120582+0000","last_clean":"2024-03-16T10:10:59.120582+0000","last_became_active":"2024-03-16T10:10:57.100493+0000","last_became_peered":"2024-03-16T10:10:57.100493+0000","last_unstale":"2024-03-16T10:10:59.120582+0000","last_undegraded":"2024-03-16T10:10:59.120582+0000","last_fullsized":"2024-03-16T10:10:59.120582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:32:53.097656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119275+0000","last_change":"2024-03-16T10:10:58.123895+0000","last_active":"2024-03-16T10:10:59.119275+0000","last_peered":"2024-03-16T10:10:59.119275+0000","last_clean":"2024-03-16T10:10:59.119275+0000","last_became_active":"2024-03-16T10:10:58.122933+0000","last_became_peered":"2024-03-16T10:10:58.122933+0000","last_unstale":"2024-03-16T10:10:59.119275+0000","last_undegraded":"2024-03-16T10:10:59.119275+0000","last_fullsized":"2024-03-16T10:10:59.119275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829089+0000","last_change":"2024-03-16T10:10:57.105506+0000","last_active":"2024-03-16T10:10:59.829089+0000","last_peered":"2024-03-16T10:10:59.829089+0000","last_clean":"2024-03-16T10:10:59.829089+0000","last_became_active":"2024-03-16T10:10:57.104984+0000","last_became_peered":"2024-03-16T10:10:57.104984+0000","last_unstale":"2024-03-16T10:10:59.829089+0000","last_undegraded":"2024-03-16T10:10:59.829089+0000","last_fullsized":"2024-03-16T10:10:59.829089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:34:00.287180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829799+0000","last_change":"2024-03-16T10:10:58.129371+0000","last_active":"2024-03-16T10:10:59.829799+0000","last_peered":"2024-03-16T10:10:59.829799+0000","last_clean":"2024-03-16T10:10:59.829799+0000","last_became_active":"2024-03-16T10:10:58.121147+0000","last_became_peered":"2024-03-16T10:10:58.121147+0000","last_unstale":"2024-03-16T10:10:59.829799+0000","last_undegraded":"2024-03-16T10:10:59.829799+0000","last_fullsized":"2024-03-16T10:10:59.829799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:09:11.898626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121588+0000","last_change":"2024-03-16T10:10:57.093832+0000","last_active":"2024-03-16T10:10:59.121588+0000","last_peered":"2024-03-16T10:10:59.121588+0000","last_clean":"2024-03-16T10:10:59.121588+0000","last_became_active":"2024-03-16T10:10:57.093278+0000","last_became_peered":"2024-03-16T10:10:57.093278+0000","last_unstale":"2024-03-16T10:10:59.121588+0000","last_undegraded":"2024-03-16T10:10:59.121588+0000","last_fullsized":"2024-03-16T10:10:59.121588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:35:54.941983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830807+0000","last_change":"2024-03-16T10:10:58.169395+0000","last_active":"2024-03-16T10:10:59.830807+0000","last_peered":"2024-03-16T10:10:59.830807+0000","last_clean":"2024-03-16T10:10:59.830807+0000","last_became_active":"2024-03-16T10:10:58.168702+0000","last_became_peered":"2024-03-16T10:10:58.168702+0000","last_unstale":"2024-03-16T10:10:59.830807+0000","last_undegraded":"2024-03-16T10:10:59.830807+0000","last_fullsized":"2024-03-16T10:10:59.830807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:13.329157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829866+0000","last_change":"2024-03-16T10:10:57.105292+0000","last_active":"2024-03-16T10:10:59.829866+0000","last_peered":"2024-03-16T10:10:59.829866+0000","last_clean":"2024-03-16T10:10:59.829866+0000","last_became_active":"2024-03-16T10:10:57.104688+0000","last_became_peered":"2024-03-16T10:10:57.104688+0000","last_unstale":"2024-03-16T10:10:59.829866+0000","last_undegraded":"2024-03-16T10:10:59.829866+0000","last_fullsized":"2024-03-16T10:10:59.829866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830999+0000","last_change":"2024-03-16T10:10:58.170384+0000","last_active":"2024-03-16T10:10:59.830999+0000","last_peered":"2024-03-16T10:10:59.830999+0000","last_clean":"2024-03-16T10:10:59.830999+0000","last_became_active":"2024-03-16T10:10:58.169787+0000","last_became_peered":"2024-03-16T10:10:58.169787+0000","last_unstale":"2024-03-16T10:10:59.830999+0000","last_undegraded":"2024-03-16T10:10:59.830999+0000","last_fullsized":"2024-03-16T10:10:59.830999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:12.960572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116762+0000","last_change":"2024-03-16T10:10:57.111549+0000","last_active":"2024-03-16T10:10:59.116762+0000","last_peered":"2024-03-16T10:10:59.116762+0000","last_clean":"2024-03-16T10:10:59.116762+0000","last_became_active":"2024-03-16T10:10:57.111022+0000","last_became_peered":"2024-03-16T10:10:57.111022+0000","last_unstale":"2024-03-16T10:10:59.116762+0000","last_undegraded":"2024-03-16T10:10:59.116762+0000","last_fullsized":"2024-03-16T10:10:59.116762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:33:03.557374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123471+0000","last_change":"2024-03-16T10:10:58.150154+0000","last_active":"2024-03-16T10:10:59.123471+0000","last_peered":"2024-03-16T10:10:59.123471+0000","last_clean":"2024-03-16T10:10:59.123471+0000","last_became_active":"2024-03-16T10:10:58.149355+0000","last_became_peered":"2024-03-16T10:10:58.149355+0000","last_unstale":"2024-03-16T10:10:59.123471+0000","last_undegraded":"2024-03-16T10:10:59.123471+0000","last_fullsized":"2024-03-16T10:10:59.123471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:27:42.059927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829400+0000","last_change":"2024-03-16T10:10:57.129962+0000","last_active":"2024-03-16T10:10:59.829400+0000","last_peered":"2024-03-16T10:10:59.829400+0000","last_clean":"2024-03-16T10:10:59.829400+0000","last_became_active":"2024-03-16T10:10:57.129583+0000","last_became_peered":"2024-03-16T10:10:57.129583+0000","last_unstale":"2024-03-16T10:10:59.829400+0000","last_undegraded":"2024-03-16T10:10:59.829400+0000","last_fullsized":"2024-03-16T10:10:59.829400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:32:53.097656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830586+0000","last_change":"2024-03-16T10:10:58.140201+0000","last_active":"2024-03-16T10:10:59.830586+0000","last_peered":"2024-03-16T10:10:59.830586+0000","last_clean":"2024-03-16T10:10:59.830586+0000","last_became_active":"2024-03-16T10:10:58.139272+0000","last_became_peered":"2024-03-16T10:10:58.139272+0000","last_unstale":"2024-03-16T10:10:59.830586+0000","last_undegraded":"2024-03-16T10:10:59.830586+0000","last_fullsized":"2024-03-16T10:10:59.830586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:27:42.059927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829599+0000","last_change":"2024-03-16T10:10:48.248686+0000","last_active":"2024-03-16T10:10:59.829599+0000","last_peered":"2024-03-16T10:10:59.829599+0000","last_clean":"2024-03-16T10:10:59.829599+0000","last_became_active":"2024-03-16T10:10:48.243054+0000","last_became_peered":"2024-03-16T10:10:48.243054+0000","last_unstale":"2024-03-16T10:10:59.829599+0000","last_undegraded":"2024-03-16T10:10:59.829599+0000","last_fullsized":"2024-03-16T10:10:59.829599+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_clean_scrub_stamp":"2024-03-16T10:10:46.517938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:42.906473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122484+0000","last_change":"2024-03-16T10:10:57.123078+0000","last_active":"2024-03-16T10:10:59.122484+0000","last_peered":"2024-03-16T10:10:59.122484+0000","last_clean":"2024-03-16T10:10:59.122484+0000","last_became_active":"2024-03-16T10:10:57.121087+0000","last_became_peered":"2024-03-16T10:10:57.121087+0000","last_unstale":"2024-03-16T10:10:59.122484+0000","last_undegraded":"2024-03-16T10:10:59.122484+0000","last_fullsized":"2024-03-16T10:10:59.122484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830505+0000","last_change":"2024-03-16T10:10:58.171123+0000","last_active":"2024-03-16T10:10:59.830505+0000","last_peered":"2024-03-16T10:10:59.830505+0000","last_clean":"2024-03-16T10:10:59.830505+0000","last_became_active":"2024-03-16T10:10:58.170849+0000","last_became_peered":"2024-03-16T10:10:58.170849+0000","last_unstale":"2024-03-16T10:10:59.830505+0000","last_undegraded":"2024-03-16T10:10:59.830505+0000","last_fullsized":"2024-03-16T10:10:59.830505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829543+0000","last_change":"2024-03-16T10:10:57.124354+0000","last_active":"2024-03-16T10:10:59.829543+0000","last_peered":"2024-03-16T10:10:59.829543+0000","last_clean":"2024-03-16T10:10:59.829543+0000","last_became_active":"2024-03-16T10:10:57.123636+0000","last_became_peered":"2024-03-16T10:10:57.123636+0000","last_unstale":"2024-03-16T10:10:59.829543+0000","last_undegraded":"2024-03-16T10:10:59.829543+0000","last_fullsized":"2024-03-16T10:10:59.829543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465205+0000","last_change":"2024-03-16T10:10:58.158168+0000","last_active":"2024-03-16T10:10:59.465205+0000","last_peered":"2024-03-16T10:10:59.465205+0000","last_clean":"2024-03-16T10:10:59.465205+0000","last_became_active":"2024-03-16T10:10:58.157882+0000","last_became_peered":"2024-03-16T10:10:58.157882+0000","last_unstale":"2024-03-16T10:10:59.465205+0000","last_undegraded":"2024-03-16T10:10:59.465205+0000","last_fullsized":"2024-03-16T10:10:59.465205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:10:11.295606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829675+0000","last_change":"2024-03-16T10:10:57.104381+0000","last_active":"2024-03-16T10:10:59.829675+0000","last_peered":"2024-03-16T10:10:59.829675+0000","last_clean":"2024-03-16T10:10:59.829675+0000","last_became_active":"2024-03-16T10:10:57.103673+0000","last_became_peered":"2024-03-16T10:10:57.103673+0000","last_unstale":"2024-03-16T10:10:59.829675+0000","last_undegraded":"2024-03-16T10:10:59.829675+0000","last_fullsized":"2024-03-16T10:10:59.829675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:45:53.158955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.115477+0000","last_change":"2024-03-16T10:10:58.136233+0000","last_active":"2024-03-16T10:10:59.115477+0000","last_peered":"2024-03-16T10:10:59.115477+0000","last_clean":"2024-03-16T10:10:59.115477+0000","last_became_active":"2024-03-16T10:10:58.135287+0000","last_became_peered":"2024-03-16T10:10:58.135287+0000","last_unstale":"2024-03-16T10:10:59.115477+0000","last_undegraded":"2024-03-16T10:10:59.115477+0000","last_fullsized":"2024-03-16T10:10:59.115477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.124008+0000","last_change":"2024-03-16T10:10:57.132978+0000","last_active":"2024-03-16T10:10:59.124008+0000","last_peered":"2024-03-16T10:10:59.124008+0000","last_clean":"2024-03-16T10:10:59.124008+0000","last_became_active":"2024-03-16T10:10:57.132199+0000","last_became_peered":"2024-03-16T10:10:57.132199+0000","last_unstale":"2024-03-16T10:10:59.124008+0000","last_undegraded":"2024-03-16T10:10:59.124008+0000","last_fullsized":"2024-03-16T10:10:59.124008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:58.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121640+0000","last_change":"2024-03-16T10:10:58.131949+0000","last_active":"2024-03-16T10:10:59.121640+0000","last_peered":"2024-03-16T10:10:59.121640+0000","last_clean":"2024-03-16T10:10:59.121640+0000","last_became_active":"2024-03-16T10:10:58.131234+0000","last_became_peered":"2024-03-16T10:10:58.131234+0000","last_unstale":"2024-03-16T10:10:59.121640+0000","last_undegraded":"2024-03-16T10:10:59.121640+0000","last_fullsized":"2024-03-16T10:10:59.121640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:59.100283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116717+0000","last_change":"2024-03-16T10:10:57.115504+0000","last_active":"2024-03-16T10:10:59.116717+0000","last_peered":"2024-03-16T10:10:59.116717+0000","last_clean":"2024-03-16T10:10:59.116717+0000","last_became_active":"2024-03-16T10:10:57.115273+0000","last_became_peered":"2024-03-16T10:10:57.115273+0000","last_unstale":"2024-03-16T10:10:59.116717+0000","last_undegraded":"2024-03-16T10:10:59.116717+0000","last_fullsized":"2024-03-16T10:10:59.116717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:58.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123414+0000","last_change":"2024-03-16T10:10:58.150495+0000","last_active":"2024-03-16T10:10:59.123414+0000","last_peered":"2024-03-16T10:10:59.123414+0000","last_clean":"2024-03-16T10:10:59.123414+0000","last_became_active":"2024-03-16T10:10:58.149680+0000","last_became_peered":"2024-03-16T10:10:58.149680+0000","last_unstale":"2024-03-16T10:10:59.123414+0000","last_undegraded":"2024-03-16T10:10:59.123414+0000","last_fullsized":"2024-03-16T10:10:59.123414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:31.763467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830902+0000","last_change":"2024-03-16T10:10:57.128751+0000","last_active":"2024-03-16T10:10:59.830902+0000","last_peered":"2024-03-16T10:10:59.830902+0000","last_clean":"2024-03-16T10:10:59.830902+0000","last_became_active":"2024-03-16T10:10:57.127909+0000","last_became_peered":"2024-03-16T10:10:57.127909+0000","last_unstale":"2024-03-16T10:10:59.830902+0000","last_undegraded":"2024-03-16T10:10:59.830902+0000","last_fullsized":"2024-03-16T10:10:59.830902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:09:33.897012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116753+0000","last_change":"2024-03-16T10:10:58.134860+0000","last_active":"2024-03-16T10:10:59.116753+0000","last_peered":"2024-03-16T10:10:59.116753+0000","last_clean":"2024-03-16T10:10:59.116753+0000","last_became_active":"2024-03-16T10:10:58.133470+0000","last_became_peered":"2024-03-16T10:10:58.133470+0000","last_unstale":"2024-03-16T10:10:59.116753+0000","last_undegraded":"2024-03-16T10:10:59.116753+0000","last_fullsized":"2024-03-16T10:10:59.116753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.151156+0000","last_change":"2024-03-16T10:10:57.110493+0000","last_active":"2024-03-16T10:10:59.151156+0000","last_peered":"2024-03-16T10:10:59.151156+0000","last_clean":"2024-03-16T10:10:59.151156+0000","last_became_active":"2024-03-16T10:10:57.106136+0000","last_became_peered":"2024-03-16T10:10:57.106136+0000","last_unstale":"2024-03-16T10:10:59.151156+0000","last_undegraded":"2024-03-16T10:10:59.151156+0000","last_fullsized":"2024-03-16T10:10:59.151156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.831141+0000","last_change":"2024-03-16T10:10:58.140570+0000","last_active":"2024-03-16T10:10:59.831141+0000","last_peered":"2024-03-16T10:10:59.831141+0000","last_clean":"2024-03-16T10:10:59.831141+0000","last_became_active":"2024-03-16T10:10:58.139532+0000","last_became_peered":"2024-03-16T10:10:58.139532+0000","last_unstale":"2024-03-16T10:10:59.831141+0000","last_undegraded":"2024-03-16T10:10:59.831141+0000","last_fullsized":"2024-03-16T10:10:59.831141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:25:29.088835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120378+0000","last_change":"2024-03-16T10:10:57.120255+0000","last_active":"2024-03-16T10:10:59.120378+0000","last_peered":"2024-03-16T10:10:59.120378+0000","last_clean":"2024-03-16T10:10:59.120378+0000","last_became_active":"2024-03-16T10:10:57.119498+0000","last_became_peered":"2024-03-16T10:10:57.119498+0000","last_unstale":"2024-03-16T10:10:59.120378+0000","last_undegraded":"2024-03-16T10:10:59.120378+0000","last_fullsized":"2024-03-16T10:10:59.120378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465935+0000","last_change":"2024-03-16T10:10:58.129362+0000","last_active":"2024-03-16T10:10:59.465935+0000","last_peered":"2024-03-16T10:10:59.465935+0000","last_clean":"2024-03-16T10:10:59.465935+0000","last_became_active":"2024-03-16T10:10:58.128580+0000","last_became_peered":"2024-03-16T10:10:58.128580+0000","last_unstale":"2024-03-16T10:10:59.465935+0000","last_undegraded":"2024-03-16T10:10:59.465935+0000","last_fullsized":"2024-03-16T10:10:59.465935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829443+0000","last_change":"2024-03-16T10:10:57.118219+0000","last_active":"2024-03-16T10:10:59.829443+0000","last_peered":"2024-03-16T10:10:59.829443+0000","last_clean":"2024-03-16T10:10:59.829443+0000","last_became_active":"2024-03-16T10:10:57.117436+0000","last_became_peered":"2024-03-16T10:10:57.117436+0000","last_unstale":"2024-03-16T10:10:59.829443+0000","last_undegraded":"2024-03-16T10:10:59.829443+0000","last_fullsized":"2024-03-16T10:10:59.829443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119531+0000","last_change":"2024-03-16T10:10:58.143411+0000","last_active":"2024-03-16T10:10:59.119531+0000","last_peered":"2024-03-16T10:10:59.119531+0000","last_clean":"2024-03-16T10:10:59.119531+0000","last_became_active":"2024-03-16T10:10:58.142517+0000","last_became_peered":"2024-03-16T10:10:58.142517+0000","last_unstale":"2024-03-16T10:10:59.119531+0000","last_undegraded":"2024-03-16T10:10:59.119531+0000","last_fullsized":"2024-03-16T10:10:59.119531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:03.131047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116378+0000","last_change":"2024-03-16T10:10:57.116431+0000","last_active":"2024-03-16T10:10:59.116378+0000","last_peered":"2024-03-16T10:10:59.116378+0000","last_clean":"2024-03-16T10:10:59.116378+0000","last_became_active":"2024-03-16T10:10:57.115761+0000","last_became_peered":"2024-03-16T10:10:57.115761+0000","last_unstale":"2024-03-16T10:10:59.116378+0000","last_undegraded":"2024-03-16T10:10:59.116378+0000","last_fullsized":"2024-03-16T10:10:59.116378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830186+0000","last_change":"2024-03-16T10:10:58.137343+0000","last_active":"2024-03-16T10:10:59.830186+0000","last_peered":"2024-03-16T10:10:59.830186+0000","last_clean":"2024-03-16T10:10:59.830186+0000","last_became_active":"2024-03-16T10:10:58.136517+0000","last_became_peered":"2024-03-16T10:10:58.136517+0000","last_unstale":"2024-03-16T10:10:59.830186+0000","last_undegraded":"2024-03-16T10:10:59.830186+0000","last_fullsized":"2024-03-16T10:10:59.830186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116455+0000","last_change":"2024-03-16T10:10:57.100812+0000","last_active":"2024-03-16T10:10:59.116455+0000","last_peered":"2024-03-16T10:10:59.116455+0000","last_clean":"2024-03-16T10:10:59.116455+0000","last_became_active":"2024-03-16T10:10:57.100312+0000","last_became_peered":"2024-03-16T10:10:57.100312+0000","last_unstale":"2024-03-16T10:10:59.116455+0000","last_undegraded":"2024-03-16T10:10:59.116455+0000","last_fullsized":"2024-03-16T10:10:59.116455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830809+0000","last_change":"2024-03-16T10:10:58.139588+0000","last_active":"2024-03-16T10:10:59.830809+0000","last_peered":"2024-03-16T10:10:59.830809+0000","last_clean":"2024-03-16T10:10:59.830809+0000","last_became_active":"2024-03-16T10:10:58.136603+0000","last_became_peered":"2024-03-16T10:10:58.136603+0000","last_unstale":"2024-03-16T10:10:59.830809+0000","last_undegraded":"2024-03-16T10:10:59.830809+0000","last_fullsized":"2024-03-16T10:10:59.830809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830698+0000","last_change":"2024-03-16T10:10:57.127697+0000","last_active":"2024-03-16T10:10:59.830698+0000","last_peered":"2024-03-16T10:10:59.830698+0000","last_clean":"2024-03-16T10:10:59.830698+0000","last_became_active":"2024-03-16T10:10:57.126922+0000","last_became_peered":"2024-03-16T10:10:57.126922+0000","last_unstale":"2024-03-16T10:10:59.830698+0000","last_undegraded":"2024-03-16T10:10:59.830698+0000","last_fullsized":"2024-03-16T10:10:59.830698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:56:47.775049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120088+0000","last_change":"2024-03-16T10:10:58.162962+0000","last_active":"2024-03-16T10:10:59.120088+0000","last_peered":"2024-03-16T10:10:59.120088+0000","last_clean":"2024-03-16T10:10:59.120088+0000","last_became_active":"2024-03-16T10:10:58.161601+0000","last_became_peered":"2024-03-16T10:10:58.161601+0000","last_unstale":"2024-03-16T10:10:59.120088+0000","last_undegraded":"2024-03-16T10:10:59.120088+0000","last_fullsized":"2024-03-16T10:10:59.120088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123098+0000","last_change":"2024-03-16T10:10:58.160830+0000","last_active":"2024-03-16T10:10:59.123098+0000","last_peered":"2024-03-16T10:10:59.123098+0000","last_clean":"2024-03-16T10:10:59.123098+0000","last_became_active":"2024-03-16T10:10:58.160108+0000","last_became_peered":"2024-03-16T10:10:58.160108+0000","last_unstale":"2024-03-16T10:10:59.123098+0000","last_undegraded":"2024-03-16T10:10:59.123098+0000","last_fullsized":"2024-03-16T10:10:59.123098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:32.891269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830094+0000","last_change":"2024-03-16T10:10:57.138309+0000","last_active":"2024-03-16T10:10:59.830094+0000","last_peered":"2024-03-16T10:10:59.830094+0000","last_clean":"2024-03-16T10:10:59.830094+0000","last_became_active":"2024-03-16T10:10:57.137445+0000","last_became_peered":"2024-03-16T10:10:57.137445+0000","last_unstale":"2024-03-16T10:10:59.830094+0000","last_undegraded":"2024-03-16T10:10:59.830094+0000","last_fullsized":"2024-03-16T10:10:59.830094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:58.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123572+0000","last_change":"2024-03-16T10:10:58.160844+0000","last_active":"2024-03-16T10:10:59.123572+0000","last_peered":"2024-03-16T10:10:59.123572+0000","last_clean":"2024-03-16T10:10:59.123572+0000","last_became_active":"2024-03-16T10:10:58.160163+0000","last_became_peered":"2024-03-16T10:10:58.160163+0000","last_unstale":"2024-03-16T10:10:59.123572+0000","last_undegraded":"2024-03-16T10:10:59.123572+0000","last_fullsized":"2024-03-16T10:10:59.123572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:08.743197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830486+0000","last_change":"2024-03-16T10:10:57.126626+0000","last_active":"2024-03-16T10:10:59.830486+0000","last_peered":"2024-03-16T10:10:59.830486+0000","last_clean":"2024-03-16T10:10:59.830486+0000","last_became_active":"2024-03-16T10:10:57.125624+0000","last_became_peered":"2024-03-16T10:10:57.125624+0000","last_unstale":"2024-03-16T10:10:59.830486+0000","last_undegraded":"2024-03-16T10:10:59.830486+0000","last_fullsized":"2024-03-16T10:10:59.830486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:33:03.557374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465968+0000","last_change":"2024-03-16T10:10:58.149370+0000","last_active":"2024-03-16T10:10:59.465968+0000","last_peered":"2024-03-16T10:10:59.465968+0000","last_clean":"2024-03-16T10:10:59.465968+0000","last_became_active":"2024-03-16T10:10:58.149011+0000","last_became_peered":"2024-03-16T10:10:58.149011+0000","last_unstale":"2024-03-16T10:10:59.465968+0000","last_undegraded":"2024-03-16T10:10:59.465968+0000","last_fullsized":"2024-03-16T10:10:59.465968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830859+0000","last_change":"2024-03-16T10:10:57.114529+0000","last_active":"2024-03-16T10:10:59.830859+0000","last_peered":"2024-03-16T10:10:59.830859+0000","last_clean":"2024-03-16T10:10:59.830859+0000","last_became_active":"2024-03-16T10:10:57.113935+0000","last_became_peered":"2024-03-16T10:10:57.113935+0000","last_unstale":"2024-03-16T10:10:59.830859+0000","last_undegraded":"2024-03-16T10:10:59.830859+0000","last_fullsized":"2024-03-16T10:10:59.830859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:34:00.287180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830836+0000","last_change":"2024-03-16T10:10:58.138174+0000","last_active":"2024-03-16T10:10:59.830836+0000","last_peered":"2024-03-16T10:10:59.830836+0000","last_clean":"2024-03-16T10:10:59.830836+0000","last_became_active":"2024-03-16T10:10:58.136767+0000","last_became_peered":"2024-03-16T10:10:58.136767+0000","last_unstale":"2024-03-16T10:10:59.830836+0000","last_undegraded":"2024-03-16T10:10:59.830836+0000","last_fullsized":"2024-03-16T10:10:59.830836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466402+0000","last_change":"2024-03-16T10:10:57.124377+0000","last_active":"2024-03-16T10:10:59.466402+0000","last_peered":"2024-03-16T10:10:59.466402+0000","last_clean":"2024-03-16T10:10:59.466402+0000","last_became_active":"2024-03-16T10:10:57.123699+0000","last_became_peered":"2024-03-16T10:10:57.123699+0000","last_unstale":"2024-03-16T10:10:59.466402+0000","last_undegraded":"2024-03-16T10:10:59.466402+0000","last_fullsized":"2024-03-16T10:10:59.466402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:32:37.635494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":660255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":660311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:11:02.627 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-16T10:11:02.628 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-16T10:11:02.628 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-16T10:11:02.628 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-16T10:11:02.628 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-16T10:11:02.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0fa02640 1 Processor -- start 2024-03-16T10:11:02.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0fa02640 1 -- start start 2024-03-16T10:11:02.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0fa02640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8e08148a30 0x7f8e08148e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0fa02640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8e080b72e0 0x7f8e08149350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0fa02640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e0806f5a0 con 0x7f8e0806e660 2024-03-16T10:11:02.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0fa02640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f8e0806fdc0 con 0x7f8e08148a30 2024-03-16T10:11:02.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0fa02640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8e080a0e20 con 0x7f8e080b72e0 2024-03-16T10:11:02.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0df9f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8e0806e660 0x7f8e08142360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:55990/0 (socket says 172.21.15.17:55990) 2024-03-16T10:11:02.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0df9f640 1 -- 172.21.15.17:0/4097744643 learned_addr learned my addr 172.21.15.17:0/4097744643 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:02.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0d79e640 1 --2- 172.21.15.17:0/4097744643 >> v2:172.21.15.174:3300/0 conn(0x7f8e08148a30 0x7f8e08148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0cf9d640 1 --2- 172.21.15.17:0/4097744643 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8e080b72e0 0x7f8e08149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 -- 172.21.15.17:0/4097744643 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1162038988 0 0) 0x7f8e0806f5a0 con 0x7f8e0806e660 2024-03-16T10:11:02.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 -- 172.21.15.17:0/4097744643 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8df4003530 con 0x7f8e0806e660 2024-03-16T10:11:02.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 -- 172.21.15.17:0/4097744643 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3401756912 0 0) 0x7f8df4003530 con 0x7f8e0806e660 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 -- 172.21.15.17:0/4097744643 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8e080b72e0 msgr2=0x7f8e08149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 --2- 172.21.15.17:0/4097744643 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8e080b72e0 0x7f8e08149350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 -- 172.21.15.17:0/4097744643 >> v2:172.21.15.174:3300/0 conn(0x7f8e08148a30 msgr2=0x7f8e08148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 --2- 172.21.15.17:0/4097744643 >> v2:172.21.15.174:3300/0 conn(0x7f8e08148a30 0x7f8e08148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 -- 172.21.15.17:0/4097744643 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e08142a70 con 0x7f8e0806e660 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 -- 172.21.15.17:0/4097744643 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f8e00001a00 con 0x7f8e0806e660 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 -- 172.21.15.17:0/4097744643 >> v1:172.21.15.17:6789/0 conn(0x7f8e0806e660 legacy=0x7f8e08142360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 --2- 172.21.15.17:0/4097744643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8df40039c0 0x7f8df4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 --2- 172.21.15.17:0/4097744643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8df4004810 0x7f8df4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 --2- 172.21.15.17:0/4097744643 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8df40051d0 0x7f8df400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 -- 172.21.15.17:0/4097744643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0806fdc0 con 0x7f8df4004810 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 -- 172.21.15.17:0/4097744643 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e080a0e20 con 0x7f8df40039c0 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8dfffff640 1 -- 172.21.15.17:0/4097744643 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8df400bf10 con 0x7f8df40051d0 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0cf9d640 1 --2- 172.21.15.17:0/4097744643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8df4004810 0x7f8df4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0d79e640 1 --2- 172.21.15.17:0/4097744643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8df40039c0 0x7f8df4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0cf9d640 1 -- 172.21.15.17:0/4097744643 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8df40051d0 msgr2=0x7f8df400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:02.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0cf9d640 1 --2- 172.21.15.17:0/4097744643 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8df40051d0 0x7f8df400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0cf9d640 1 -- 172.21.15.17:0/4097744643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8df40039c0 msgr2=0x7f8df4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:02.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0cf9d640 1 --2- 172.21.15.17:0/4097744643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8df40039c0 0x7f8df4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.781+0000 7f8e0cf9d640 1 -- 172.21.15.17:0/4097744643 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df4003530 con 0x7f8df4004810 2024-03-16T10:11:02.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0cf9d640 1 --2- 172.21.15.17:0/4097744643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8df4004810 0x7f8df4004c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009ab0 tx=0x7f8df801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9166b57d96c8f04 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:02.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8dfffff640 1 -- 172.21.15.17:0/4097744643 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df80b2070 con 0x7f8df4004810 2024-03-16T10:11:02.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8dfffff640 1 -- 172.21.15.17:0/4097744643 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df80a5570 con 0x7f8df4004810 2024-03-16T10:11:02.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8dfffff640 1 -- 172.21.15.17:0/4097744643 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df80096b0 con 0x7f8df4004810 2024-03-16T10:11:02.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0fa02640 1 -- 172.21.15.17:0/4097744643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8df4004810 msgr2=0x7f8df4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0fa02640 1 --2- 172.21.15.17:0/4097744643 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8df4004810 0x7f8df4004c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009ab0 tx=0x7f8df801ca90 comp rx=0 tx=0).stop 2024-03-16T10:11:02.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0df9f640 1 -- 172.21.15.17:0/4097744643 reap_dead start 2024-03-16T10:11:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0fa02640 1 -- 172.21.15.17:0/4097744643 shutdown_connections 2024-03-16T10:11:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0fa02640 1 -- 172.21.15.17:0/4097744643 >> 172.21.15.17:0/4097744643 conn(0x7f8e080718a0 msgr2=0x7f8e08071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0fa02640 1 -- 172.21.15.17:0/4097744643 shutdown_connections 2024-03-16T10:11:02.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0fa02640 1 -- 172.21.15.17:0/4097744643 wait complete. 2024-03-16T10:11:02.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0fa02640 1 Processor -- start 2024-03-16T10:11:02.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0fa02640 1 -- start start 2024-03-16T10:11:02.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0fa02640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e081db330 0x7f8e08146f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0fa02640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e08147480 0x7f8e0814ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0fa02640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8e0814b040 0x7f8e081de5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0fa02640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0806cdc0 con 0x7f8e08147480 2024-03-16T10:11:02.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0fa02640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e080a0fa0 con 0x7f8e081db330 2024-03-16T10:11:02.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0fa02640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0806cf40 con 0x7f8e0814b040 2024-03-16T10:11:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0cf9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e08147480 0x7f8e0814ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0cf9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e08147480 0x7f8e0814ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33596/0 (socket says 172.21.15.17:33596) 2024-03-16T10:11:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.785+0000 7f8e0cf9d640 1 -- 172.21.15.17:0/2581584228 learned_addr learned my addr 172.21.15.17:0/2581584228 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0d79e640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e081db330 0x7f8e08146f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0df9f640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8e0814b040 0x7f8e081de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0cf9d640 1 -- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8e0814b040 msgr2=0x7f8e081de5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0cf9d640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8e0814b040 0x7f8e081de5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0cf9d640 1 -- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e081db330 msgr2=0x7f8e08146f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0cf9d640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e081db330 0x7f8e08146f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0cf9d640 1 -- 172.21.15.17:0/2581584228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e081478e0 con 0x7f8e08147480 2024-03-16T10:11:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0df9f640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8e0814b040 0x7f8e081de5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0d79e640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e081db330 0x7f8e08146f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0cf9d640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e08147480 0x7f8e0814ab00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8df801c8c0 tx=0x7f8df8019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:02.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8dfdffb640 1 -- 172.21.15.17:0/2581584228 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df80ba020 con 0x7f8e08147480 2024-03-16T10:11:02.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8dfdffb640 1 -- 172.21.15.17:0/2581584228 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8df80ad440 con 0x7f8e08147480 2024-03-16T10:11:02.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0fa02640 1 -- 172.21.15.17:0/2581584228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e081db710 con 0x7f8e08147480 2024-03-16T10:11:02.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8dfdffb640 1 -- 172.21.15.17:0/2581584228 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df80b2040 con 0x7f8e08147480 2024-03-16T10:11:02.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0fa02640 1 -- 172.21.15.17:0/2581584228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e08146740 con 0x7f8e08147480 2024-03-16T10:11:02.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0fa02640 1 -- 172.21.15.17:0/2581584228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e0806a130 con 0x7f8e08147480 2024-03-16T10:11:02.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8dfdffb640 1 -- 172.21.15.17:0/2581584228 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8df80bd070 con 0x7f8e08147480 2024-03-16T10:11:02.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8dfdffb640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8dec067660 0x7f8dec069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:02.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8dfdffb640 1 -- 172.21.15.17:0/2581584228 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8df811e860 con 0x7f8e08147480 2024-03-16T10:11:02.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0d79e640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8dec067660 0x7f8dec069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:02.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.789+0000 7f8e0d79e640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8dec067660 0x7f8dec069b20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009b70 tx=0x7f8e04008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:02.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.793+0000 7f8dfdffb640 1 -- 172.21.15.17:0/2581584228 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8df80167a0 con 0x7f8e08147480 2024-03-16T10:11:02.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.997+0000 7f8e0fa02640 1 -- 172.21.15.17:0/2581584228 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f8e08067a70 con 0x7f8e08147480 2024-03-16T10:11:03.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:02.997+0000 7f8dfdffb640 1 -- 172.21.15.17:0/2581584228 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+166533 (secure 0 0 0) 0x7f8df811e090 con 0x7f8e08147480 2024-03-16T10:11:03.004 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:03.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.001+0000 7f8e0fa02640 1 -- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8dec067660 msgr2=0x7f8dec069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.001+0000 7f8e0fa02640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8dec067660 0x7f8dec069b20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8e04009b70 tx=0x7f8e04008040 comp rx=0 tx=0).stop 2024-03-16T10:11:03.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.001+0000 7f8e0fa02640 1 -- 172.21.15.17:0/2581584228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e08147480 msgr2=0x7f8e0814ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.001+0000 7f8e0fa02640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e08147480 0x7f8e0814ab00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8df801c8c0 tx=0x7f8df8019830 comp rx=0 tx=0).stop 2024-03-16T10:11:03.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.001+0000 7f8e0fa02640 1 -- 172.21.15.17:0/2581584228 shutdown_connections 2024-03-16T10:11:03.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.001+0000 7f8e0fa02640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8dec067660 0x7f8dec069b20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.001+0000 7f8e0fa02640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8e0814b040 0x7f8e081de5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.001+0000 7f8e0fa02640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e08147480 0x7f8e0814ab00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.001+0000 7f8e0fa02640 1 --2- 172.21.15.17:0/2581584228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e081db330 0x7f8e08146f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.001+0000 7f8e0fa02640 1 -- 172.21.15.17:0/2581584228 >> 172.21.15.17:0/2581584228 conn(0x7f8e080718a0 msgr2=0x7f8e080721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.001+0000 7f8e0fa02640 1 -- 172.21.15.17:0/2581584228 shutdown_connections 2024-03-16T10:11:03.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.001+0000 7f8e0fa02640 1 -- 172.21.15.17:0/2581584228 wait complete. 2024-03-16T10:11:03.023 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":2142122996},{"type":"v1","addr":"172.21.15.174:6833","nonce":2142122996}]},"active_addr":"172.21.15.174:6833/2142122996","active_change":"2024-03-16T10:10:37.999910+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4108,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":445624298}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":1943678911}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":3808983575}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.174:0","nonce":3632576967}]}]} 2024-03-16T10:11:03.026 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-16T10:11:03.026 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-16T10:11:03.026 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:03.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.137+0000 7fa8fc99c640 1 Processor -- start 2024-03-16T10:11:03.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.137+0000 7fa8fc99c640 1 -- start start 2024-03-16T10:11:03.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.137+0000 7fa8fc99c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8f80b72e0 0x7fa8f8142360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8fc99c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8f806f5a0 con 0x7fa8f806e660 2024-03-16T10:11:03.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8fc99c640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8f806fdc0 con 0x7fa8f8148a30 2024-03-16T10:11:03.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8fc99c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f80a0e20 con 0x7fa8f80b72e0 2024-03-16T10:11:03.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f6d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa8f806e660 0x7fa8f814fb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56000/0 (socket says 172.21.15.17:56000) 2024-03-16T10:11:03.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f6d9d640 1 -- 172.21.15.17:0/90871158 learned_addr learned my addr 172.21.15.17:0/90871158 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:03.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f5d9b640 1 --2- 172.21.15.17:0/90871158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8f80b72e0 0x7fa8f8142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 -- 172.21.15.17:0/90871158 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1269187230 0 0) 0x7fa8f806f5a0 con 0x7fa8f806e660 2024-03-16T10:11:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 -- 172.21.15.17:0/90871158 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8e4003530 con 0x7fa8f806e660 2024-03-16T10:11:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 -- 172.21.15.17:0/90871158 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3133773422 0 0) 0x7fa8e4003530 con 0x7fa8f806e660 2024-03-16T10:11:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 -- 172.21.15.17:0/90871158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8f80b72e0 msgr2=0x7fa8f8142360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 --2- 172.21.15.17:0/90871158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8f80b72e0 0x7fa8f8142360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 -- 172.21.15.17:0/90871158 >> v1:172.21.15.174:6789/0 conn(0x7fa8f8148a30 legacy=0x7fa8f8148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 -- 172.21.15.17:0/90871158 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f8149520 con 0x7fa8f806e660 2024-03-16T10:11:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 -- 172.21.15.17:0/90871158 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fa8f0001a00 con 0x7fa8f806e660 2024-03-16T10:11:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 -- 172.21.15.17:0/90871158 >> v1:172.21.15.17:6789/0 conn(0x7fa8f806e660 legacy=0x7fa8f814fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 --2- 172.21.15.17:0/90871158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8e40039f0 0x7fa8e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 --2- 172.21.15.17:0/90871158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8e4004310 0x7fa8e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 --2- 172.21.15.17:0/90871158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e4005060 0x7fa8e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 -- 172.21.15.17:0/90871158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f80a0e20 con 0x7fa8e4005060 2024-03-16T10:11:03.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 -- 172.21.15.17:0/90871158 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e400bf40 con 0x7fa8e4004310 2024-03-16T10:11:03.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f559a640 1 -- 172.21.15.17:0/90871158 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa8e400c140 con 0x7fa8e40039f0 2024-03-16T10:11:03.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f6d9d640 1 --2- 172.21.15.17:0/90871158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e4005060 0x7fa8e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f5d9b640 1 --2- 172.21.15.17:0/90871158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8e4004310 0x7fa8e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f659c640 1 --2- 172.21.15.17:0/90871158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8e40039f0 0x7fa8e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f6d9d640 1 -- 172.21.15.17:0/90871158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8e40039f0 msgr2=0x7fa8e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:03.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f6d9d640 1 --2- 172.21.15.17:0/90871158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8e40039f0 0x7fa8e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f6d9d640 1 -- 172.21.15.17:0/90871158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8e4004310 msgr2=0x7fa8e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:03.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f6d9d640 1 --2- 172.21.15.17:0/90871158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8e4004310 0x7fa8e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f6d9d640 1 -- 172.21.15.17:0/90871158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e4003530 con 0x7fa8e4005060 2024-03-16T10:11:03.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.141+0000 7fa8f6d9d640 1 --2- 172.21.15.17:0/90871158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e4005060 0x7fa8e400b9d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8f806ea60 tx=0x7fa8f001d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7334e6e3df732853 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f559a640 1 -- 172.21.15.17:0/90871158 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f000a450 con 0x7fa8e4005060 2024-03-16T10:11:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f559a640 1 -- 172.21.15.17:0/90871158 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f00072d0 con 0x7fa8e4005060 2024-03-16T10:11:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f559a640 1 -- 172.21.15.17:0/90871158 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f00076c0 con 0x7fa8e4005060 2024-03-16T10:11:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8fc99c640 1 -- 172.21.15.17:0/90871158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e4005060 msgr2=0x7fa8e400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8fc99c640 1 --2- 172.21.15.17:0/90871158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8e4005060 0x7fa8e400b9d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa8f806ea60 tx=0x7fa8f001d2a0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f6d9d640 1 -- 172.21.15.17:0/90871158 reap_dead start 2024-03-16T10:11:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8fc99c640 1 -- 172.21.15.17:0/90871158 shutdown_connections 2024-03-16T10:11:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8fc99c640 1 -- 172.21.15.17:0/90871158 >> 172.21.15.17:0/90871158 conn(0x7fa8f80718a0 msgr2=0x7fa8f8071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8fc99c640 1 -- 172.21.15.17:0/90871158 shutdown_connections 2024-03-16T10:11:03.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8fc99c640 1 -- 172.21.15.17:0/90871158 wait complete. 2024-03-16T10:11:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8fc99c640 1 Processor -- start 2024-03-16T10:11:03.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8fc99c640 1 -- start start 2024-03-16T10:11:03.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8fc99c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8f8147a40 0x7fa8f81440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8fc99c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f8144620 0x7fa8f80b9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8fc99c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8f80ba110 0x7fa8f80ba570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8fc99c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f8142570 con 0x7fa8f80ba110 2024-03-16T10:11:03.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8fc99c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f80a0fa0 con 0x7fa8f8144620 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8fc99c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa8f81426f0 con 0x7fa8f8147a40 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8f80ba110 0x7fa8f80ba570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8f80ba110 0x7fa8f80ba570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33616/0 (socket says 172.21.15.17:33616) 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f6d9d640 1 -- 172.21.15.17:0/659636602 learned_addr learned my addr 172.21.15.17:0/659636602 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f659c640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8f8147a40 0x7fa8f81440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f5d9b640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f8144620 0x7fa8f80b9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f6d9d640 1 -- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8f8147a40 msgr2=0x7fa8f81440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f6d9d640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8f8147a40 0x7fa8f81440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f6d9d640 1 -- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f8144620 msgr2=0x7fa8f80b9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f6d9d640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f8144620 0x7fa8f80b9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f6d9d640 1 -- 172.21.15.17:0/659636602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f80b28c0 con 0x7fa8f80ba110 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.145+0000 7fa8f659c640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8f8147a40 0x7fa8f81440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.149+0000 7fa8f6d9d640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8f80ba110 0x7fa8f80ba570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0098cc0 tx=0x7fa8e4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.149+0000 7fa8eeffd640 1 -- 172.21.15.17:0/659636602 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f000a450 con 0x7fa8f80ba110 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.149+0000 7fa8eeffd640 1 -- 172.21.15.17:0/659636602 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f00070d0 con 0x7fa8f80ba110 2024-03-16T10:11:03.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.149+0000 7fa8eeffd640 1 -- 172.21.15.17:0/659636602 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0007480 con 0x7fa8f80ba110 2024-03-16T10:11:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.149+0000 7fa8fc99c640 1 -- 172.21.15.17:0/659636602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f8144b50 con 0x7fa8f80ba110 2024-03-16T10:11:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.149+0000 7fa8fc99c640 1 -- 172.21.15.17:0/659636602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f8145440 con 0x7fa8f80ba110 2024-03-16T10:11:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.149+0000 7fa8fc99c640 1 -- 172.21.15.17:0/659636602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8f806a130 con 0x7fa8f80ba110 2024-03-16T10:11:03.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.149+0000 7fa8eeffd640 1 -- 172.21.15.17:0/659636602 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa8f00bd030 con 0x7fa8f80ba110 2024-03-16T10:11:03.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.149+0000 7fa8eeffd640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa8d8067690 0x7fa8d8069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.149+0000 7fa8eeffd640 1 -- 172.21.15.17:0/659636602 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa8f011f5e0 con 0x7fa8f80ba110 2024-03-16T10:11:03.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.149+0000 7fa8f659c640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa8d8067690 0x7fa8d8069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.153+0000 7fa8f659c640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa8d8067690 0x7fa8d8069b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8007990 tx=0x7fa8e8007900 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.153+0000 7fa8eeffd640 1 -- 172.21.15.17:0/659636602 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa8f00c4060 con 0x7fa8f80ba110 2024-03-16T10:11:03.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.309+0000 7fa8fc99c640 1 -- 172.21.15.17:0/659636602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8f80b2b30 con 0x7fa8f80ba110 2024-03-16T10:11:03.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.313+0000 7fa8eeffd640 1 -- 172.21.15.17:0/659636602 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7fa8f0016fb0 con 0x7fa8f80ba110 2024-03-16T10:11:03.316 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:03.317 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:03.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.313+0000 7fa8fc99c640 1 -- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa8d8067690 msgr2=0x7fa8d8069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.313+0000 7fa8fc99c640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa8d8067690 0x7fa8d8069b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8007990 tx=0x7fa8e8007900 comp rx=0 tx=0).stop 2024-03-16T10:11:03.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.313+0000 7fa8fc99c640 1 -- 172.21.15.17:0/659636602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8f80ba110 msgr2=0x7fa8f80ba570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.313+0000 7fa8fc99c640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8f80ba110 0x7fa8f80ba570 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0098cc0 tx=0x7fa8e4005840 comp rx=0 tx=0).stop 2024-03-16T10:11:03.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.313+0000 7fa8fc99c640 1 -- 172.21.15.17:0/659636602 shutdown_connections 2024-03-16T10:11:03.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.313+0000 7fa8fc99c640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa8d8067690 0x7fa8d8069b50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.313+0000 7fa8fc99c640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa8f80ba110 0x7fa8f80ba570 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.313+0000 7fa8fc99c640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f8144620 0x7fa8f80b9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.313+0000 7fa8fc99c640 1 --2- 172.21.15.17:0/659636602 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa8f8147a40 0x7fa8f81440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.313+0000 7fa8fc99c640 1 -- 172.21.15.17:0/659636602 >> 172.21.15.17:0/659636602 conn(0x7fa8f80718a0 msgr2=0x7fa8f8148e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.313+0000 7fa8fc99c640 1 -- 172.21.15.17:0/659636602 shutdown_connections 2024-03-16T10:11:03.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.313+0000 7fa8fc99c640 1 -- 172.21.15.17:0/659636602 wait complete. 2024-03-16T10:11:03.334 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-16T10:11:03.334 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:03.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.469+0000 7f82a53f1640 1 Processor -- start 2024-03-16T10:11:03.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.469+0000 7f82a53f1640 1 -- start start 2024-03-16T10:11:03.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.473+0000 7f82a53f1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f82a006d750 0x7f82a00ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.473+0000 7f82a53f1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82a014b0d0 0x7f82a0151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.473+0000 7f82a53f1640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82a006db30 con 0x7f82a00b2830 2024-03-16T10:11:03.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.473+0000 7f82a53f1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f82a00b5170 con 0x7f82a006d750 2024-03-16T10:11:03.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.473+0000 7f82a53f1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f82a00aec90 con 0x7f82a014b0d0 2024-03-16T10:11:03.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.473+0000 7f829effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f82a006d750 0x7f82a00ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.473+0000 7f829effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f82a006d750 0x7f82a00ae750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33624/0 (socket says 172.21.15.17:33624) 2024-03-16T10:11:03.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.473+0000 7f829effd640 1 -- 172.21.15.17:0/497815929 learned_addr learned my addr 172.21.15.17:0/497815929 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:03.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.473+0000 7f829effd640 1 -- 172.21.15.17:0/497815929 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82a014b0d0 msgr2=0x7f82a0151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:03.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.473+0000 7f829effd640 1 --2- 172.21.15.17:0/497815929 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82a014b0d0 0x7f82a0151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.473+0000 7f829effd640 1 -- 172.21.15.17:0/497815929 >> v1:172.21.15.174:6789/0 conn(0x7f82a00b2830 legacy=0x7f82a014a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:03.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.473+0000 7f829effd640 1 -- 172.21.15.17:0/497815929 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a00b4b20 con 0x7f82a006d750 2024-03-16T10:11:03.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f829effd640 1 --2- 172.21.15.17:0/497815929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82a006d750 0x7f82a00ae750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f82a006de60 tx=0x7f829401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9a3a6aee7afc2e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f829dffb640 1 -- 172.21.15.17:0/497815929 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82940b0070 con 0x7f82a006d750 2024-03-16T10:11:03.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f829dffb640 1 -- 172.21.15.17:0/497815929 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82940ab600 con 0x7f82a006d750 2024-03-16T10:11:03.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f829dffb640 1 -- 172.21.15.17:0/497815929 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82940ab9d0 con 0x7f82a006d750 2024-03-16T10:11:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 -- 172.21.15.17:0/497815929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82a006d750 msgr2=0x7f82a00ae750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 --2- 172.21.15.17:0/497815929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82a006d750 0x7f82a00ae750 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f82a006de60 tx=0x7f829401c450 comp rx=0 tx=0).stop 2024-03-16T10:11:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 -- 172.21.15.17:0/497815929 shutdown_connections 2024-03-16T10:11:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 --2- 172.21.15.17:0/497815929 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82a014b0d0 0x7f82a0151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 --2- 172.21.15.17:0/497815929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82a006d750 0x7f82a00ae750 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 -- 172.21.15.17:0/497815929 >> 172.21.15.17:0/497815929 conn(0x7f82a00b55c0 msgr2=0x7f82a00b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 -- 172.21.15.17:0/497815929 shutdown_connections 2024-03-16T10:11:03.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 -- 172.21.15.17:0/497815929 wait complete. 2024-03-16T10:11:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 Processor -- start 2024-03-16T10:11:03.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 -- start start 2024-03-16T10:11:03.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82a006d750 0x7f82a0137bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82a00b2830 0x7f82a0138130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82a014b0d0 0x7f82a01e3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f82a0154ba0 con 0x7f82a006d750 2024-03-16T10:11:03.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f82a0154a20 con 0x7f82a014b0d0 2024-03-16T10:11:03.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.477+0000 7f82a53f1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f82a0154d20 con 0x7f82a00b2830 2024-03-16T10:11:03.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f829effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82a006d750 0x7f82a0137bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f829effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82a006d750 0x7f82a0137bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33634/0 (socket says 172.21.15.17:33634) 2024-03-16T10:11:03.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f829effd640 1 -- 172.21.15.17:0/1944396560 learned_addr learned my addr 172.21.15.17:0/1944396560 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:03.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f829e7fc640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82a00b2830 0x7f82a0138130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f829f7fe640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82a014b0d0 0x7f82a01e3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f829effd640 1 -- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82a00b2830 msgr2=0x7f82a0138130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f829effd640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82a00b2830 0x7f82a0138130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f829effd640 1 -- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82a014b0d0 msgr2=0x7f82a01e3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f829effd640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82a014b0d0 0x7f82a01e3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f829effd640 1 -- 172.21.15.17:0/1944396560 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a013e760 con 0x7f82a006d750 2024-03-16T10:11:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f829f7fe640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82a014b0d0 0x7f82a01e3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f829effd640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82a006d750 0x7f82a0137bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82940ae040 tx=0x7f829401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f82877fe640 1 -- 172.21.15.17:0/1944396560 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82940ba020 con 0x7f82a006d750 2024-03-16T10:11:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f82877fe640 1 -- 172.21.15.17:0/1944396560 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82940b6620 con 0x7f82a006d750 2024-03-16T10:11:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f82a53f1640 1 -- 172.21.15.17:0/1944396560 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a0140030 con 0x7f82a006d750 2024-03-16T10:11:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f82877fe640 1 -- 172.21.15.17:0/1944396560 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82940b0040 con 0x7f82a006d750 2024-03-16T10:11:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f82a53f1640 1 -- 172.21.15.17:0/1944396560 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a013fad0 con 0x7f82a006d750 2024-03-16T10:11:03.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f82a53f1640 1 -- 172.21.15.17:0/1944396560 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a0068360 con 0x7f82a006d750 2024-03-16T10:11:03.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f82877fe640 1 -- 172.21.15.17:0/1944396560 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f82940bd020 con 0x7f82a006d750 2024-03-16T10:11:03.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f82877fe640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f828c067690 0x7f828c069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f82877fe640 1 -- 172.21.15.17:0/1944396560 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f829411fbb0 con 0x7f82a006d750 2024-03-16T10:11:03.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f829e7fc640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f828c067690 0x7f828c069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.481+0000 7f829e7fc640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f828c067690 0x7f828c069b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f82a013d930 tx=0x7f8290009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.485+0000 7f82877fe640 1 -- 172.21.15.17:0/1944396560 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f82940c4030 con 0x7f82a006d750 2024-03-16T10:11:03.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.645+0000 7f82a53f1640 1 -- 172.21.15.17:0/1944396560 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82a0140200 con 0x7f82a006d750 2024-03-16T10:11:03.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.645+0000 7f82877fe640 1 -- 172.21.15.17:0/1944396560 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7f82940160d0 con 0x7f82a006d750 2024-03-16T10:11:03.650 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:03.650 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:03.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.649+0000 7f82a53f1640 1 -- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f828c067690 msgr2=0x7f828c069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.649+0000 7f82a53f1640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f828c067690 0x7f828c069b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f82a013d930 tx=0x7f8290009040 comp rx=0 tx=0).stop 2024-03-16T10:11:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.649+0000 7f82a53f1640 1 -- 172.21.15.17:0/1944396560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82a006d750 msgr2=0x7f82a0137bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.649+0000 7f82a53f1640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82a006d750 0x7f82a0137bf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f82940ae040 tx=0x7f829401ac90 comp rx=0 tx=0).stop 2024-03-16T10:11:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.649+0000 7f82a53f1640 1 -- 172.21.15.17:0/1944396560 shutdown_connections 2024-03-16T10:11:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.649+0000 7f82a53f1640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f828c067690 0x7f828c069b50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.649+0000 7f82a53f1640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82a014b0d0 0x7f82a01e3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.649+0000 7f82a53f1640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82a00b2830 0x7f82a0138130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.649+0000 7f82a53f1640 1 --2- 172.21.15.17:0/1944396560 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f82a006d750 0x7f82a0137bf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.649+0000 7f82a53f1640 1 -- 172.21.15.17:0/1944396560 >> 172.21.15.17:0/1944396560 conn(0x7f82a00b55c0 msgr2=0x7f82a014a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.649+0000 7f82a53f1640 1 -- 172.21.15.17:0/1944396560 shutdown_connections 2024-03-16T10:11:03.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.649+0000 7f82a53f1640 1 -- 172.21.15.17:0/1944396560 wait complete. 2024-03-16T10:11:03.668 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-16T10:11:03.669 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-16T10:11:03.669 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-16T10:11:03.669 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-16T10:11:03.670 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-16T10:11:03.670 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-16T10:11:03.671 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-16T10:11:03.671 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-16T10:11:03.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.849+0000 7f0ab8b97640 1 Processor -- start 2024-03-16T10:11:03.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.853+0000 7f0ab8b97640 1 -- start start 2024-03-16T10:11:03.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.853+0000 7f0ab8b97640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0ab40b60e0 0x7f0ab406d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.853+0000 7f0ab8b97640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab413e100 0x7f0ab413e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.853+0000 7f0ab8b97640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ab40b0190 con 0x7f0ab40aed90 2024-03-16T10:11:03.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.853+0000 7f0ab8b97640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f0ab40b5b10 con 0x7f0ab40b60e0 2024-03-16T10:11:03.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.853+0000 7f0ab8b97640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab40afc70 con 0x7f0ab413e100 2024-03-16T10:11:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.853+0000 7f0ab259c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0ab40b60e0 0x7f0ab406d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.853+0000 7f0ab1d9b640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f0ab40aed90 0x7f0ab40af330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.17:58618/0 (socket says 172.21.15.17:58618) 2024-03-16T10:11:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.853+0000 7f0ab1d9b640 1 -- 172.21.15.17:0/17450621 learned_addr learned my addr 172.21.15.17:0/17450621 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.853+0000 7f0ab2d9d640 1 --2- 172.21.15.17:0/17450621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab413e100 0x7f0ab413e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.853+0000 7f0ab259c640 1 -- 172.21.15.17:0/17450621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab413e100 msgr2=0x7f0ab413e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:03.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.853+0000 7f0ab259c640 1 --2- 172.21.15.17:0/17450621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab413e100 0x7f0ab413e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.853+0000 7f0ab259c640 1 -- 172.21.15.17:0/17450621 >> v1:172.21.15.174:6789/0 conn(0x7f0ab40aed90 legacy=0x7f0ab40af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.853+0000 7f0ab259c640 1 -- 172.21.15.17:0/17450621 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab4070a00 con 0x7f0ab40b60e0 2024-03-16T10:11:03.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f0ab259c640 1 --2- 172.21.15.17:0/17450621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab40b60e0 0x7f0ab406d810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80097c0 tx=0x7f0aa801c2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcbc03c6c44bc1e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f0ab159a640 1 -- 172.21.15.17:0/17450621 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80b0070 con 0x7f0ab40b60e0 2024-03-16T10:11:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f0ab159a640 1 -- 172.21.15.17:0/17450621 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa80ab600 con 0x7f0ab40b60e0 2024-03-16T10:11:03.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f0ab159a640 1 -- 172.21.15.17:0/17450621 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80ab9d0 con 0x7f0ab40b60e0 2024-03-16T10:11:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f0ab8b97640 1 -- 172.21.15.17:0/17450621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab40b60e0 msgr2=0x7f0ab406d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f0ab8b97640 1 --2- 172.21.15.17:0/17450621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab40b60e0 0x7f0ab406d810 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80097c0 tx=0x7f0aa801c2c0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f0ab8b97640 1 -- 172.21.15.17:0/17450621 shutdown_connections 2024-03-16T10:11:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f0ab8b97640 1 --2- 172.21.15.17:0/17450621 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab413e100 0x7f0ab413e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f0ab8b97640 1 --2- 172.21.15.17:0/17450621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab40b60e0 0x7f0ab406d810 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f0ab8b97640 1 -- 172.21.15.17:0/17450621 >> 172.21.15.17:0/17450621 conn(0x7f0ab40b9a30 msgr2=0x7f0ab40b9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f0ab8b97640 1 -- 172.21.15.17:0/17450621 shutdown_connections 2024-03-16T10:11:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f0ab8b97640 1 -- 172.21.15.17:0/17450621 wait complete. 2024-03-16T10:11:03.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f261eeed640 1 Processor -- start 2024-03-16T10:11:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab8b97640 1 Processor -- start 2024-03-16T10:11:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f261eeed640 1 -- start start 2024-03-16T10:11:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f261eeed640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f261806d750 0x7f2618072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f261eeed640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f26180affc0 0x7f26180b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f261eeed640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f261806db30 con 0x7f26180b1170 2024-03-16T10:11:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f261eeed640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f2618073380 con 0x7f26180affc0 2024-03-16T10:11:03.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.857+0000 7f261eeed640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f26180b4c30 con 0x7f261806d750 2024-03-16T10:11:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f261d48a640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f26180b1170 0x7f2618145660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56016/0 (socket says 172.21.15.17:56016) 2024-03-16T10:11:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f261d48a640 1 -- 172.21.15.17:0/3847022123 learned_addr learned my addr 172.21.15.17:0/3847022123 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f2617fff640 1 --2- 172.21.15.17:0/3847022123 >> v2:172.21.15.17:3300/0 conn(0x7f26180affc0 0x7f26180b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab8b97640 1 -- start start 2024-03-16T10:11:03.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f261cc89640 1 --2- 172.21.15.17:0/3847022123 >> v2:172.21.15.174:3300/0 conn(0x7f261806d750 0x7f2618072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab8b97640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab40aed90 0x7f0ab41449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab8b97640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ab40b60e0 0x7f0ab40b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab8b97640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab413e100 0x7f0ab40b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab8b97640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab406be70 con 0x7f0ab40b60e0 2024-03-16T10:11:03.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab8b97640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab406bcf0 con 0x7f0ab40aed90 2024-03-16T10:11:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab8b97640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0ab406bff0 con 0x7f0ab413e100 2024-03-16T10:11:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab1d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ab40b60e0 0x7f0ab40b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab40aed90 0x7f0ab41449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab1d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ab40b60e0 0x7f0ab40b31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33646/0 (socket says 172.21.15.17:33646) 2024-03-16T10:11:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab1d9b640 1 -- 172.21.15.17:0/2171641251 learned_addr learned my addr 172.21.15.17:0/2171641251 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab2d9d640 1 --2- 172.21.15.17:0/2171641251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab413e100 0x7f0ab40b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f2617fff640 1 -- 172.21.15.17:0/3847022123 >> v2:172.21.15.174:3300/0 conn(0x7f261806d750 msgr2=0x7f2618072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f2617fff640 1 --2- 172.21.15.17:0/3847022123 >> v2:172.21.15.174:3300/0 conn(0x7f261806d750 0x7f2618072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f2617fff640 1 -- 172.21.15.17:0/3847022123 >> v1:172.21.15.17:6789/0 conn(0x7f26180b1170 legacy=0x7f2618145660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f2617fff640 1 -- 172.21.15.17:0/3847022123 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26180b4f10 con 0x7f26180affc0 2024-03-16T10:11:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab1d9b640 1 -- 172.21.15.17:0/2171641251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab413e100 msgr2=0x7f0ab40b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab1d9b640 1 --2- 172.21.15.17:0/2171641251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab413e100 0x7f0ab40b0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab1d9b640 1 -- 172.21.15.17:0/2171641251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab40aed90 msgr2=0x7f0ab41449f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab1d9b640 1 --2- 172.21.15.17:0/2171641251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab40aed90 0x7f0ab41449f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab1d9b640 1 -- 172.21.15.17:0/2171641251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab413c290 con 0x7f0ab40b60e0 2024-03-16T10:11:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab2d9d640 1 --2- 172.21.15.17:0/2171641251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0ab413e100 0x7f0ab40b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab1d9b640 1 --2- 172.21.15.17:0/2171641251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ab40b60e0 0x7f0ab40b31f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400c400 tx=0x7f0aa400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0aa2ffd640 1 -- 172.21.15.17:0/2171641251 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4017070 con 0x7f0ab40b60e0 2024-03-16T10:11:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0aa2ffd640 1 -- 172.21.15.17:0/2171641251 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa4005cf0 con 0x7f0ab40b60e0 2024-03-16T10:11:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7ff0b0ef0640 1 Processor -- start 2024-03-16T10:11:03.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7ff0b0ef0640 1 -- start start 2024-03-16T10:11:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7ff0b0ef0640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff0ac0b5c00 0x7ff0ac0b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7ff0b0ef0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0ac14b670 con 0x7ff0ac14c3e0 2024-03-16T10:11:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7ff0b0ef0640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0ac0b5fe0 con 0x7ff0ac0b0db0 2024-03-16T10:11:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7ff0b0ef0640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ff0ac0b2620 con 0x7ff0ac0b5c00 2024-03-16T10:11:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7ff0aad9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff0ac14c3e0 0x7ff0ac155f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56026/0 (socket says 172.21.15.17:56026) 2024-03-16T10:11:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7ff0aad9d640 1 -- 172.21.15.17:0/1182191583 learned_addr learned my addr 172.21.15.17:0/1182191583 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab8b97640 1 -- 172.21.15.17:0/2171641251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab413afd0 con 0x7f0ab40b60e0 2024-03-16T10:11:03.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0ab8b97640 1 -- 172.21.15.17:0/2171641251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab40b1fb0 con 0x7f0ab40b60e0 2024-03-16T10:11:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f0aa2ffd640 1 -- 172.21.15.17:0/2171641251 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40072d0 con 0x7f0ab40b60e0 2024-03-16T10:11:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7ff0aa59c640 1 --2- 172.21.15.17:0/1182191583 >> v2:172.21.15.174:3300/0 conn(0x7ff0ac0b5c00 0x7ff0ac0b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f2617fff640 1 --2- 172.21.15.17:0/3847022123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26180affc0 0x7f26180b0c30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f261806de30 tx=0x7f261001c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54ff845152c7e3d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f26177fe640 1 -- 172.21.15.17:0/3847022123 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100b2070 con 0x7f26180affc0 2024-03-16T10:11:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f26177fe640 1 -- 172.21.15.17:0/3847022123 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26100a5820 con 0x7f26180affc0 2024-03-16T10:11:03.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f26177fe640 1 -- 172.21.15.17:0/3847022123 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100a5bf0 con 0x7f26180affc0 2024-03-16T10:11:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7ff0aa59c640 1 -- 172.21.15.17:0/1182191583 >> v1:172.21.15.174:6789/0 conn(0x7ff0ac0b0db0 legacy=0x7ff0ac14bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7ff0aa59c640 1 -- 172.21.15.17:0/1182191583 >> v1:172.21.15.17:6789/0 conn(0x7ff0ac14c3e0 legacy=0x7ff0ac155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7ff0aa59c640 1 -- 172.21.15.17:0/1182191583 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac0b2850 con 0x7ff0ac0b5c00 2024-03-16T10:11:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f0ab8b97640 1 -- 172.21.15.17:0/2171641251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ab406a130 con 0x7f0ab40b60e0 2024-03-16T10:11:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f0aa2ffd640 1 -- 172.21.15.17:0/2171641251 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f0aa401a070 con 0x7f0ab40b60e0 2024-03-16T10:11:03.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f261eeed640 1 -- 172.21.15.17:0/3847022123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26180affc0 msgr2=0x7f26180b0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f261eeed640 1 --2- 172.21.15.17:0/3847022123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26180affc0 0x7f26180b0c30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f261806de30 tx=0x7f261001c7c0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f0aa2ffd640 1 --2- 172.21.15.17:0/2171641251 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0a9c067870 0x7f0a9c069d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f0aa2ffd640 1 -- 172.21.15.17:0/2171641251 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0aa4084b10 con 0x7f0ab40b60e0 2024-03-16T10:11:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f0aa2ffd640 1 --2- 172.21.15.17:0/2171641251 >> [v2:172.21.15.17:6803/1266087291,v1:172.21.15.17:6806/1266087291] conn(0x7f0a9c06fa70 0x7f0a9c071e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f0aa2ffd640 1 -- 172.21.15.17:0/2171641251 --> [v2:172.21.15.17:6803/1266087291,v1:172.21.15.17:6806/1266087291] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0aa4012e50 con 0x7f0a9c06fa70 2024-03-16T10:11:03.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f83ec8fa640 1 Processor -- start 2024-03-16T10:11:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.861+0000 7f83ec8fa640 1 -- start start 2024-03-16T10:11:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f0ab2d9d640 1 --2- 172.21.15.17:0/2171641251 >> [v2:172.21.15.17:6803/1266087291,v1:172.21.15.17:6806/1266087291] conn(0x7f0a9c06fa70 0x7f0a9c071e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f0ab259c640 1 --2- 172.21.15.17:0/2171641251 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0a9c067870 0x7f0a9c069d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f0aa2ffd640 1 -- 172.21.15.17:0/2171641251 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0aa405b070 con 0x7f0ab40b60e0 2024-03-16T10:11:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f261eeed640 1 -- 172.21.15.17:0/3847022123 shutdown_connections 2024-03-16T10:11:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f261eeed640 1 --2- 172.21.15.17:0/3847022123 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26180affc0 0x7f26180b0c30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f261eeed640 1 --2- 172.21.15.17:0/3847022123 >> v2:172.21.15.174:3300/0 conn(0x7f261806d750 0x7f2618072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f261eeed640 1 -- 172.21.15.17:0/3847022123 >> 172.21.15.17:0/3847022123 conn(0x7f26180b55c0 msgr2=0x7f26180b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7ff0aa59c640 1 --2- 172.21.15.17:0/1182191583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0ac0b5c00 0x7ff0ac0b0870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0097c0 tx=0x7ff09c01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc982e4d63b2e023 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f261eeed640 1 -- 172.21.15.17:0/3847022123 shutdown_connections 2024-03-16T10:11:03.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f261eeed640 1 -- 172.21.15.17:0/3847022123 wait complete. 2024-03-16T10:11:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f0ab259c640 1 --2- 172.21.15.17:0/2171641251 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0a9c067870 0x7f0a9c069d30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80ad000 tx=0x7f0aa80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f0ab2d9d640 1 --2- 172.21.15.17:0/2171641251 >> [v2:172.21.15.17:6803/1266087291,v1:172.21.15.17:6806/1266087291] conn(0x7f0a9c06fa70 0x7f0a9c071e90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f261eeed640 1 Processor -- start 2024-03-16T10:11:03.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f261eeed640 1 -- start start 2024-03-16T10:11:03.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f0aa2ffd640 1 -- 172.21.15.17:0/2171641251 <== osd.3 v2:172.21.15.17:6803/1266087291 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0aa4012e50 con 0x7f0a9c06fa70 2024-03-16T10:11:03.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7ff0a9d9b640 1 -- 172.21.15.17:0/1182191583 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0b0070 con 0x7ff0ac0b5c00 2024-03-16T10:11:03.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7ff0a9d9b640 1 -- 172.21.15.17:0/1182191583 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09c0ab600 con 0x7ff0ac0b5c00 2024-03-16T10:11:03.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7ff0a9d9b640 1 -- 172.21.15.17:0/1182191583 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0ab9d0 con 0x7ff0ac0b5c00 2024-03-16T10:11:03.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83ec8fa640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f83e80b5540 0x7f83e80afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83ec8fa640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e80b04f0 0x7f83e80b0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83ec8fa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f83e80b3280 0x7f83e80b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83ec8fa640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f83e80731f0 con 0x7f83e80b3280 2024-03-16T10:11:03.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83ec8fa640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f83e80b4690 con 0x7f83e80b5540 2024-03-16T10:11:03.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83ec8fa640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f83e80b1df0 con 0x7f83e80b04f0 2024-03-16T10:11:03.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83e6d9d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f83e80b5540 0x7f83e80afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83e6d9d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f83e80b5540 0x7f83e80afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.17:40044/0 (socket says 172.21.15.17:40044) 2024-03-16T10:11:03.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83e759e640 1 --2- 172.21.15.17:0/3330271193 >> v2:172.21.15.17:3300/0 conn(0x7f83e80b3280 0x7f83e80b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83e6d9d640 1 -- 172.21.15.17:0/3330271193 learned_addr learned my addr 172.21.15.17:0/3330271193 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83e659c640 1 --2- 172.21.15.17:0/3330271193 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e80b04f0 0x7f83e80b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83e759e640 1 -- 172.21.15.17:0/3330271193 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e80b04f0 msgr2=0x7f83e80b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83e759e640 1 --2- 172.21.15.17:0/3330271193 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e80b04f0 0x7f83e80b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83e759e640 1 -- 172.21.15.17:0/3330271193 >> v2:172.21.15.174:3300/0 conn(0x7f83e80b5540 msgr2=0x7f83e80afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83e759e640 1 --2- 172.21.15.17:0/3330271193 >> v2:172.21.15.174:3300/0 conn(0x7f83e80b5540 0x7f83e80afd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.865+0000 7f83e759e640 1 -- 172.21.15.17:0/3330271193 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e80b2d20 con 0x7f83e80b3280 2024-03-16T10:11:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/1182191583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0ac0b5c00 msgr2=0x7ff0ac0b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7ff0b0ef0640 1 --2- 172.21.15.17:0/1182191583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0ac0b5c00 0x7ff0ac0b0870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0097c0 tx=0x7ff09c01c330 comp rx=0 tx=0).stop 2024-03-16T10:11:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f83e759e640 1 --2- 172.21.15.17:0/3330271193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b3280 0x7f83e80b36f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83e80b4420 tx=0x7f83dc02d2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368047bb65cc358 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/1182191583 shutdown_connections 2024-03-16T10:11:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7ff0b0ef0640 1 --2- 172.21.15.17:0/1182191583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0ac0b5c00 0x7ff0ac0b0870 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0097c0 tx=0x7ff09c01c330 comp rx=0 tx=0).stop 2024-03-16T10:11:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/1182191583 >> 172.21.15.17:0/1182191583 conn(0x7ff0ac06b460 msgr2=0x7ff0ac06b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f261eeed640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f261806d750 0x7f26181482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f261eeed640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26180affc0 0x7f26180abed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f261eeed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26180b1170 0x7f2618138b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f261eeed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2618145ef0 con 0x7f26180b1170 2024-03-16T10:11:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f261eeed640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2618145d70 con 0x7f26180affc0 2024-03-16T10:11:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f261eeed640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2618146070 con 0x7f261806d750 2024-03-16T10:11:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f83e5d9b640 1 -- 172.21.15.17:0/3330271193 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0c1070 con 0x7f83e80b3280 2024-03-16T10:11:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f2617fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26180affc0 0x7f26180abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f2617fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26180affc0 0x7f26180abed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.17:40048/0 (socket says 172.21.15.17:40048) 2024-03-16T10:11:03.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f2617fff640 1 -- 172.21.15.17:0/3545818904 learned_addr learned my addr 172.21.15.17:0/3545818904 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:03.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f261cc89640 1 --2- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f261806d750 0x7f26181482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/1182191583 shutdown_connections 2024-03-16T10:11:03.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f83e5d9b640 1 -- 172.21.15.17:0/3330271193 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83dc00d2d0 con 0x7f83e80b3280 2024-03-16T10:11:03.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f2617fff640 1 -- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f261806d750 msgr2=0x7f26181482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f2617fff640 1 --2- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f261806d750 0x7f26181482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f2617fff640 1 -- 172.21.15.17:0/3545818904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26180b1170 msgr2=0x7f2618138b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f2617fff640 1 --2- 172.21.15.17:0/3545818904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f26180b1170 0x7f2618138b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7f2617fff640 1 -- 172.21.15.17:0/3545818904 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2618144fc0 con 0x7f26180affc0 2024-03-16T10:11:03.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83e5d9b640 1 -- 172.21.15.17:0/3330271193 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc00b670 con 0x7f83e80b3280 2024-03-16T10:11:03.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f72b5f640 1 Processor -- start 2024-03-16T10:11:03.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f2617fff640 1 --2- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26180affc0 0x7f26180abed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f261001a3f0 tx=0x7f2610019560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f26157fa640 1 -- 172.21.15.17:0/3545818904 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100bd020 con 0x7f26180affc0 2024-03-16T10:11:03.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f261eeed640 1 -- 172.21.15.17:0/3545818904 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26180ab9b0 con 0x7f26180affc0 2024-03-16T10:11:03.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f72b5f640 1 -- start start 2024-03-16T10:11:03.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.869+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/1182191583 wait complete. 2024-03-16T10:11:03.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0b0ef0640 1 Processor -- start 2024-03-16T10:11:03.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f72b5f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f6c0b1370 0x7f8f6c0b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f72b5f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8f6c0b5380 0x7f8f6c0b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f72b5f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f6c0b2650 con 0x7f8f6c0b3f30 2024-03-16T10:11:03.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f72b5f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8f6c0b3280 con 0x7f8f6c0b5380 2024-03-16T10:11:03.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f72b5f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6c13d9f0 con 0x7f8f6c0b1370 2024-03-16T10:11:03.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f708fb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f6c0b1370 0x7f8f6c0b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3330271193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b3280 msgr2=0x7f83e80b36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 --2- 172.21.15.17:0/3330271193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b3280 0x7f83e80b36f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83e80b4420 tx=0x7f83dc02d2f0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f261eeed640 1 -- 172.21.15.17:0/3545818904 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26180a94b0 con 0x7f26180affc0 2024-03-16T10:11:03.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f26157fa640 1 -- 172.21.15.17:0/3545818904 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26100a5e50 con 0x7f26180affc0 2024-03-16T10:11:03.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f26157fa640 1 -- 172.21.15.17:0/3545818904 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26100b2040 con 0x7f26180affc0 2024-03-16T10:11:03.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3330271193 shutdown_connections 2024-03-16T10:11:03.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 --2- 172.21.15.17:0/3330271193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b3280 0x7f83e80b36f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 --2- 172.21.15.17:0/3330271193 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e80b04f0 0x7f83e80b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 --2- 172.21.15.17:0/3330271193 >> v2:172.21.15.174:3300/0 conn(0x7f83e80b5540 0x7f83e80afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3330271193 >> 172.21.15.17:0/3330271193 conn(0x7f83e806b290 msgr2=0x7f83e806b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3330271193 shutdown_connections 2024-03-16T10:11:03.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3330271193 wait complete. 2024-03-16T10:11:03.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f710fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8f6c0b3f30 0x7f8f6c146ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56034/0 (socket says 172.21.15.17:56034) 2024-03-16T10:11:03.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f710fc640 1 -- 172.21.15.17:0/4261646388 learned_addr learned my addr 172.21.15.17:0/4261646388 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:03.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0b0ef0640 1 -- start start 2024-03-16T10:11:03.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0b0ef0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff0ac0b0db0 0x7ff0ac14f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0b0ef0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0ac146050 0x7ff0ac144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0b0ef0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0ac148cc0 0x7ff0ac1356a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0b0ef0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac0afd80 con 0x7ff0ac146050 2024-03-16T10:11:03.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0b0ef0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac0b52a0 con 0x7ff0ac148cc0 2024-03-16T10:11:03.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0b0ef0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ac0aff00 con 0x7ff0ac0b0db0 2024-03-16T10:11:03.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 Processor -- start 2024-03-16T10:11:03.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f6b7fe640 1 -- 172.21.15.17:0/4261646388 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4173674927 0 0) 0x7f8f6c0b2650 con 0x7f8f6c0b3f30 2024-03-16T10:11:03.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f6b7fe640 1 -- 172.21.15.17:0/4261646388 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f54003530 con 0x7f8f6c0b3f30 2024-03-16T10:11:03.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0aad9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0ac148cc0 0x7ff0ac1356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0aad9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0ac148cc0 0x7ff0ac1356a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.17:40058/0 (socket says 172.21.15.17:40058) 2024-03-16T10:11:03.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0aad9d640 1 -- 172.21.15.17:0/2605683738 learned_addr learned my addr 172.21.15.17:0/2605683738 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:03.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 -- start start 2024-03-16T10:11:03.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0a1d9b640 1 --2- 172.21.15.17:0/2605683738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0ac146050 0x7ff0ac144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f708fb640 1 -- 172.21.15.17:0/4261646388 >> v2:172.21.15.17:3300/0 conn(0x7f8f6c0b5380 msgr2=0x7f8f6c0b39f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-16T10:11:03.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f6b7fe640 1 -- 172.21.15.17:0/4261646388 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2074391166 0 0) 0x7f8f54003530 con 0x7f8f6c0b3f30 2024-03-16T10:11:03.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f6bfff640 1 --2- 172.21.15.17:0/4261646388 >> v2:172.21.15.17:3300/0 conn(0x7f8f6c0b5380 0x7f8f6c0b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0aad9d640 1 -- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff0ac0b0db0 msgr2=0x7ff0ac14f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0aad9d640 1 --2- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff0ac0b0db0 0x7ff0ac14f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0aad9d640 1 -- 172.21.15.17:0/2605683738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0ac146050 msgr2=0x7ff0ac144190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0aad9d640 1 --2- 172.21.15.17:0/2605683738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0ac146050 0x7ff0ac144190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0aad9d640 1 -- 172.21.15.17:0/2605683738 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac143990 con 0x7ff0ac148cc0 2024-03-16T10:11:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7ff0a1d9b640 1 --2- 172.21.15.17:0/2605683738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0ac146050 0x7ff0ac144190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:11:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7ff0aad9d640 1 --2- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0ac148cc0 0x7ff0ac1356a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400ed40 tx=0x7ff0a400ce90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7fa6159c5640 1 Processor -- start 2024-03-16T10:11:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7fa6159c5640 1 -- start start 2024-03-16T10:11:03.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f26157fa640 1 -- 172.21.15.17:0/3545818904 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f26100c0020 con 0x7f26180affc0 2024-03-16T10:11:03.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7fa6159c5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa61008b2d0 0x7fa610157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7fa6159c5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6100aee90 con 0x7fa6100b46c0 2024-03-16T10:11:03.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7fa6159c5640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6100b4aa0 con 0x7fa610158130 2024-03-16T10:11:03.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7fa6159c5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa6100af3e0 con 0x7fa61008b2d0 2024-03-16T10:11:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7fa6149c3640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa6100b46c0 0x7fa6100add90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56044/0 (socket says 172.21.15.17:56044) 2024-03-16T10:11:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7fa6149c3640 1 -- 172.21.15.17:0/4073896586 learned_addr learned my addr 172.21.15.17:0/4073896586 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7fa60f7fe640 1 -- 172.21.15.17:0/4073896586 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2106144568 0 0) 0x7fa6100aee90 con 0x7fa6100b46c0 2024-03-16T10:11:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7fa60ffff640 1 --2- 172.21.15.17:0/4073896586 >> v2:172.21.15.17:3300/0 conn(0x7fa61008b2d0 0x7fa610157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7fa60f7fe640 1 -- 172.21.15.17:0/4073896586 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5fc003dc0 con 0x7fa6100b46c0 2024-03-16T10:11:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7fa60f7fe640 1 -- 172.21.15.17:0/4073896586 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2292806020 0 0) 0x7fa6100b4aa0 con 0x7fa610158130 2024-03-16T10:11:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7fa60f7fe640 1 -- 172.21.15.17:0/4073896586 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6100aee90 con 0x7fa610158130 2024-03-16T10:11:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7fa60ffff640 1 -- 172.21.15.17:0/4073896586 >> v1:172.21.15.174:6789/0 conn(0x7fa610158130 legacy=0x7fa61015eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7fa60ffff640 1 -- 172.21.15.17:0/4073896586 >> v1:172.21.15.17:6789/0 conn(0x7fa6100b46c0 legacy=0x7fa6100add90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7fa60ffff640 1 -- 172.21.15.17:0/4073896586 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61009de20 con 0x7fa61008b2d0 2024-03-16T10:11:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7ff0a37fe640 1 -- 172.21.15.17:0/2605683738 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4017630 con 0x7ff0ac148cc0 2024-03-16T10:11:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/2605683738 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ac143cd0 con 0x7ff0ac148cc0 2024-03-16T10:11:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/2605683738 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ac0a9900 con 0x7ff0ac148cc0 2024-03-16T10:11:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f261eeed640 1 -- 172.21.15.17:0/3545818904 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f25f4001040 con 0x7f26180affc0 2024-03-16T10:11:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f26157fa640 1 --2- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f25fc067660 0x7f25fc069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f708fb640 1 --2- 172.21.15.17:0/4261646388 >> v2:172.21.15.17:3300/0 conn(0x7f8f6c0b5380 0x7f8f6c0b39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f708fb640 1 -- 172.21.15.17:0/4261646388 >> v1:172.21.15.17:6789/0 conn(0x7f8f6c0b3f30 legacy=0x7f8f6c146ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f8f708fb640 1 -- 172.21.15.17:0/4261646388 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c0af7c0 con 0x7f8f6c0b1370 2024-03-16T10:11:03.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f8f6bfff640 1 --2- 172.21.15.17:0/4261646388 >> v2:172.21.15.17:3300/0 conn(0x7f8f6c0b5380 0x7f8f6c0b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:03.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b04f0 0x7f83e81592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e80b5540 0x7f83e80aeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e81ec7b0 0x7f83e815ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f83e80b1a70 con 0x7f83e80b04f0 2024-03-16T10:11:03.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83e80b4040 con 0x7f83e80b5540 2024-03-16T10:11:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.873+0000 7f83ec8fa640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f83e8157dd0 con 0x7f83e81ec7b0 2024-03-16T10:11:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f83e759e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e81ec7b0 0x7f83e815ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f83e6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b04f0 0x7f83e81592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f83e659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e80b5540 0x7f83e80aeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f83e6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b04f0 0x7f83e81592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33700/0 (socket says 172.21.15.17:33700) 2024-03-16T10:11:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f83e6d9d640 1 -- 172.21.15.17:0/3685809552 learned_addr learned my addr 172.21.15.17:0/3685809552 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f83e659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e80b5540 0x7f83e80aeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.17:40060/0 (socket says 172.21.15.17:40060) 2024-03-16T10:11:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f83e6d9d640 1 -- 172.21.15.17:0/3685809552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e81ec7b0 msgr2=0x7f83e815ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f83e6d9d640 1 --2- 172.21.15.17:0/3685809552 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f83e81ec7b0 0x7f83e815ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f83e6d9d640 1 -- 172.21.15.17:0/3685809552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e80b5540 msgr2=0x7f83e80aeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f83e6d9d640 1 --2- 172.21.15.17:0/3685809552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e80b5540 0x7f83e80aeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f83e6d9d640 1 -- 172.21.15.17:0/3685809552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e8158410 con 0x7f83e80b04f0 2024-03-16T10:11:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f83e6d9d640 1 --2- 172.21.15.17:0/3685809552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b04f0 0x7f83e81592e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f83e0008a50 tx=0x7f83e000e4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f261cc89640 1 --2- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f25fc067660 0x7f25fc069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f26157fa640 1 -- 172.21.15.17:0/3545818904 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f26101208d0 con 0x7f26180affc0 2024-03-16T10:11:03.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7ff0a37fe640 1 -- 172.21.15.17:0/2605683738 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a40177d0 con 0x7ff0ac148cc0 2024-03-16T10:11:03.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f26157fa640 1 --2- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:6819/326911164,v1:172.21.15.174:6822/326911164] conn(0x7f25fc06f860 0x7f25fc071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f26157fa640 1 -- 172.21.15.17:0/3545818904 --> [v2:172.21.15.174:6819/326911164,v1:172.21.15.174:6822/326911164] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25fc072330 con 0x7f25fc06f860 2024-03-16T10:11:03.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f26157fa640 1 -- 172.21.15.17:0/3545818904 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f26100ed0e0 con 0x7f26180affc0 2024-03-16T10:11:03.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f8f708fb640 1 --2- 172.21.15.17:0/4261646388 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f6c0b1370 0x7f8f6c0b5cc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009d90 tx=0x7f8f6401c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faf41533716acbb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f261d48a640 1 --2- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:6819/326911164,v1:172.21.15.174:6822/326911164] conn(0x7f25fc06f860 0x7f25fc071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f8f6b7fe640 1 -- 172.21.15.17:0/4261646388 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f640ad070 con 0x7f8f6c0b1370 2024-03-16T10:11:03.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7ff0a37fe640 1 -- 172.21.15.17:0/2605683738 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4017c00 con 0x7ff0ac148cc0 2024-03-16T10:11:03.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/2605683738 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff0ac068360 con 0x7ff0ac148cc0 2024-03-16T10:11:03.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f261cc89640 1 --2- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f25fc067660 0x7f25fc069b20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f26181494a0 tx=0x7f260c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f8f6b7fe640 1 -- 172.21.15.17:0/4261646388 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f640a9410 con 0x7f8f6c0b1370 2024-03-16T10:11:03.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7f8f6b7fe640 1 -- 172.21.15.17:0/4261646388 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c001680 con 0x7f8f6c0b1370 2024-03-16T10:11:03.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7fa60ffff640 1 --2- 172.21.15.17:0/4073896586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61008b2d0 0x7fa610157bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa600007bc0 tx=0x7fa6000253c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baadd8ddec3d16d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.877+0000 7ff0a37fe640 1 -- 172.21.15.17:0/2605683738 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7ff0a4009070 con 0x7ff0ac148cc0 2024-03-16T10:11:03.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 -- 172.21.15.17:0/4261646388 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f6c0b1370 msgr2=0x7f8f6c0b5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 --2- 172.21.15.17:0/4261646388 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f6c0b1370 0x7f8f6c0b5cc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8f64009d90 tx=0x7f8f6401c820 comp rx=0 tx=0).stop 2024-03-16T10:11:03.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f83cf7fe640 1 -- 172.21.15.17:0/3685809552 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0014460 con 0x7f83e80b04f0 2024-03-16T10:11:03.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f83cf7fe640 1 -- 172.21.15.17:0/3685809552 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e0019070 con 0x7f83e80b04f0 2024-03-16T10:11:03.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f83cf7fe640 1 -- 172.21.15.17:0/3685809552 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0014800 con 0x7f83e80b04f0 2024-03-16T10:11:03.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f261d48a640 1 --2- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:6819/326911164,v1:172.21.15.174:6822/326911164] conn(0x7f25fc06f860 0x7f25fc071c80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 -- 172.21.15.17:0/4261646388 shutdown_connections 2024-03-16T10:11:03.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 --2- 172.21.15.17:0/4261646388 >> v2:172.21.15.17:3300/0 conn(0x7f8f6c0b5380 0x7f8f6c0b39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 --2- 172.21.15.17:0/4261646388 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f6c0b1370 0x7f8f6c0b5cc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 -- 172.21.15.17:0/4261646388 >> 172.21.15.17:0/4261646388 conn(0x7f8f6c06b460 msgr2=0x7f8f6c06b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 -- 172.21.15.17:0/4261646388 shutdown_connections 2024-03-16T10:11:03.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7fa60f7fe640 1 -- 172.21.15.17:0/4073896586 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000ba070 con 0x7fa61008b2d0 2024-03-16T10:11:03.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7fa60f7fe640 1 -- 172.21.15.17:0/4073896586 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6000b5600 con 0x7fa61008b2d0 2024-03-16T10:11:03.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7fa60f7fe640 1 -- 172.21.15.17:0/4073896586 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6000b59d0 con 0x7fa61008b2d0 2024-03-16T10:11:03.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7fa6159c5640 1 -- 172.21.15.17:0/4073896586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61008b2d0 msgr2=0x7fa610157bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7fa6159c5640 1 --2- 172.21.15.17:0/4073896586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61008b2d0 0x7fa610157bf0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa600007bc0 tx=0x7fa6000253c0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7fa6159c5640 1 -- 172.21.15.17:0/4073896586 shutdown_connections 2024-03-16T10:11:03.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7fa6159c5640 1 --2- 172.21.15.17:0/4073896586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa61008b2d0 0x7fa610157bf0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7fa6159c5640 1 -- 172.21.15.17:0/4073896586 >> 172.21.15.17:0/4073896586 conn(0x7fa6100b5550 msgr2=0x7fa6100b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3685809552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e815dc00 con 0x7f83e80b04f0 2024-03-16T10:11:03.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3685809552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e815a690 con 0x7f83e80b04f0 2024-03-16T10:11:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 -- 172.21.15.17:0/4261646388 wait complete. 2024-03-16T10:11:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 Processor -- start 2024-03-16T10:11:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7fa6159c5640 1 -- 172.21.15.17:0/4073896586 shutdown_connections 2024-03-16T10:11:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7fa6159c5640 1 -- 172.21.15.17:0/4073896586 wait complete. 2024-03-16T10:11:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f26157fa640 1 -- 172.21.15.17:0/3545818904 <== osd.4 v2:172.21.15.174:6819/326911164 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f25fc072330 con 0x7f25fc06f860 2024-03-16T10:11:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3685809552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83e8068360 con 0x7f83e80b04f0 2024-03-16T10:11:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 -- start start 2024-03-16T10:11:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f6c0b1370 0x7f8f6c13c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f6c0b3f30 0x7f8f6c1e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c0b5380 0x7f8f6c1ec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c14cc90 con 0x7f8f6c0b5380 2024-03-16T10:11:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c0b50f0 con 0x7f8f6c0b1370 2024-03-16T10:11:03.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f72b5f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6c148370 con 0x7f8f6c0b3f30 2024-03-16T10:11:03.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7fa6159c5640 1 Processor -- start 2024-03-16T10:11:03.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f83cf7fe640 1 -- 172.21.15.17:0/3685809552 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f83e00149a0 con 0x7f83e80b04f0 2024-03-16T10:11:03.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f6c0b3f30 0x7f8f6c1e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f6bfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f6c0b3f30 0x7f8f6c1e97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.17:49460/0 (socket says 172.21.15.17:49460) 2024-03-16T10:11:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f6bfff640 1 -- 172.21.15.17:0/2235953556 learned_addr learned my addr 172.21.15.17:0/2235953556 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7f8f710fc640 1 --2- 172.21.15.17:0/2235953556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c0b5380 0x7f8f6c1ec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f83cf7fe640 1 --2- 172.21.15.17:0/3685809552 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f83d40676b0 0x7f83d4069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f83cf7fe640 1 -- 172.21.15.17:0/3685809552 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f83e0088500 con 0x7f83e80b04f0 2024-03-16T10:11:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f83cf7fe640 1 --2- 172.21.15.17:0/3685809552 >> [v2:172.21.15.174:6803/4101740318,v1:172.21.15.174:6805/4101740318] conn(0x7f83d406f930 0x7f83d4071d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f83cf7fe640 1 -- 172.21.15.17:0/3685809552 --> [v2:172.21.15.174:6803/4101740318,v1:172.21.15.174:6805/4101740318] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83e0010e50 con 0x7f83d406f930 2024-03-16T10:11:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f83cf7fe640 1 -- 172.21.15.17:0/3685809552 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f83e0088900 con 0x7f83e80b04f0 2024-03-16T10:11:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.881+0000 7fa6159c5640 1 -- start start 2024-03-16T10:11:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6159c5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa61008b2d0 0x7fa6100b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6159c5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6100b46c0 0x7fa610083510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6159c5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa610158130 0x7fa6100b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6159c5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa6100b29e0 con 0x7fa610158130 2024-03-16T10:11:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6159c5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6100b2860 con 0x7fa61008b2d0 2024-03-16T10:11:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6159c5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa610162df0 con 0x7fa6100b46c0 2024-03-16T10:11:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f8f710fc640 1 -- 172.21.15.17:0/2235953556 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f6c0b3f30 msgr2=0x7f8f6c1e97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f8f710fc640 1 --2- 172.21.15.17:0/2235953556 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f6c0b3f30 0x7f8f6c1e97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f8f710fc640 1 -- 172.21.15.17:0/2235953556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f6c0b1370 msgr2=0x7f8f6c13c220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:11:03.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f8f710fc640 1 --2- 172.21.15.17:0/2235953556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f6c0b1370 0x7f8f6c13c220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f8f710fc640 1 -- 172.21.15.17:0/2235953556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c141240 con 0x7f8f6c0b5380 2024-03-16T10:11:03.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f83e759e640 1 --2- 172.21.15.17:0/3685809552 >> [v2:172.21.15.174:6803/4101740318,v1:172.21.15.174:6805/4101740318] conn(0x7f83d406f930 0x7f83d4071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6149c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa61008b2d0 0x7fa6100b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6149c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa61008b2d0 0x7fa6100b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.17:40074/0 (socket says 172.21.15.17:40074) 2024-03-16T10:11:03.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6149c3640 1 -- 172.21.15.17:0/781878902 learned_addr learned my addr 172.21.15.17:0/781878902 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:03.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa60ffff640 1 --2- 172.21.15.17:0/781878902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6100b46c0 0x7fa610083510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f8f710fc640 1 --2- 172.21.15.17:0/2235953556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c0b5380 0x7f8f6c1ec5c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c001390 tx=0x7f8f5c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6149c3640 1 -- 172.21.15.17:0/781878902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6100b46c0 msgr2=0x7fa610083510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6149c3640 1 --2- 172.21.15.17:0/781878902 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa6100b46c0 0x7fa610083510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6149c3640 1 -- 172.21.15.17:0/781878902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa610158130 msgr2=0x7fa6100b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6149c3640 1 --2- 172.21.15.17:0/781878902 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa610158130 0x7fa6100b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6149c3640 1 -- 172.21.15.17:0/781878902 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6100af900 con 0x7fa61008b2d0 2024-03-16T10:11:03.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f8f697fa640 1 -- 172.21.15.17:0/2235953556 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c017070 con 0x7f8f6c0b5380 2024-03-16T10:11:03.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f8f697fa640 1 -- 172.21.15.17:0/2235953556 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f5c005b00 con 0x7f8f6c0b5380 2024-03-16T10:11:03.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f83e759e640 1 --2- 172.21.15.17:0/3685809552 >> [v2:172.21.15.174:6803/4101740318,v1:172.21.15.174:6805/4101740318] conn(0x7f83d406f930 0x7f83d4071d50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f8f72b5f640 1 -- 172.21.15.17:0/2235953556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6c141820 con 0x7f8f6c0b5380 2024-03-16T10:11:03.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f8f72b5f640 1 -- 172.21.15.17:0/2235953556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f6c13ca20 con 0x7f8f6c0b5380 2024-03-16T10:11:03.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7fa6149c3640 1 --2- 172.21.15.17:0/781878902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa61008b2d0 0x7fa6100b1290 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800c4f0 tx=0x7fa5f800c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f8f72b5f640 1 -- 172.21.15.17:0/2235953556 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f6c142760 con 0x7f8f6c0b5380 2024-03-16T10:11:03.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f83cf7fe640 1 -- 172.21.15.17:0/3685809552 <== osd.6 v2:172.21.15.174:6803/4101740318 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83e0010e50 con 0x7f83d406f930 2024-03-16T10:11:03.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7f83e659c640 1 --2- 172.21.15.17:0/3685809552 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f83d40676b0 0x7f83d4069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7f8f697fa640 1 -- 172.21.15.17:0/2235953556 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c007070 con 0x7f8f6c0b5380 2024-03-16T10:11:03.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7f8f697fa640 1 -- 172.21.15.17:0/2235953556 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8f5c0072d0 con 0x7f8f6c0b5380 2024-03-16T10:11:03.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7fa60d7fa640 1 -- 172.21.15.17:0/781878902 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8017070 con 0x7fa61008b2d0 2024-03-16T10:11:03.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7fa6159c5640 1 -- 172.21.15.17:0/781878902 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6100b0ad0 con 0x7fa61008b2d0 2024-03-16T10:11:03.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7fa60d7fa640 1 -- 172.21.15.17:0/781878902 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f8005cf0 con 0x7fa61008b2d0 2024-03-16T10:11:03.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7fa60d7fa640 1 -- 172.21.15.17:0/781878902 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8007270 con 0x7fa61008b2d0 2024-03-16T10:11:03.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7fa6159c5640 1 -- 172.21.15.17:0/781878902 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6100b0d70 con 0x7fa61008b2d0 2024-03-16T10:11:03.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7f83e659c640 1 --2- 172.21.15.17:0/3685809552 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f83d40676b0 0x7f83d4069b70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f83e80b4420 tx=0x7f83d8009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7f8f697fa640 1 --2- 172.21.15.17:0/2235953556 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8f58067660 0x7f8f58069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7fa6159c5640 1 -- 172.21.15.17:0/781878902 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa610075520 con 0x7fa61008b2d0 2024-03-16T10:11:03.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7f8f697fa640 1 -- 172.21.15.17:0/2235953556 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8f5c084ba0 con 0x7f8f6c0b5380 2024-03-16T10:11:03.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7f8f708fb640 1 --2- 172.21.15.17:0/2235953556 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8f58067660 0x7f8f58069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7f8f697fa640 1 --2- 172.21.15.17:0/2235953556 >> [v2:172.21.15.17:6824/3540530625,v1:172.21.15.17:6825/3540530625] conn(0x7f8f5806f860 0x7f8f58071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7f8f6bfff640 1 --2- 172.21.15.17:0/2235953556 >> [v2:172.21.15.17:6824/3540530625,v1:172.21.15.17:6825/3540530625] conn(0x7f8f5806f860 0x7f8f58071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7f8f6bfff640 1 --2- 172.21.15.17:0/2235953556 >> [v2:172.21.15.17:6824/3540530625,v1:172.21.15.17:6825/3540530625] conn(0x7f8f5806f860 0x7f8f58071c80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8f708fb640 1 --2- 172.21.15.17:0/2235953556 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8f58067660 0x7f8f58069b20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8f640af000 tx=0x7f8f640a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f874d32f640 1 Processor -- start 2024-03-16T10:11:03.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f874d32f640 1 -- start start 2024-03-16T10:11:03.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f874d32f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8740035fc0 0x7f8740037800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f874d32f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87400363a0 con 0x7f87400ea970 2024-03-16T10:11:03.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f874d32f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8740035820 con 0x7f87400eadb0 2024-03-16T10:11:03.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.885+0000 7f874d32f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f87400ea450 con 0x7f8740035fc0 2024-03-16T10:11:03.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7f8747fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8740035fc0 0x7f8740037800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7f8747fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8740035fc0 0x7f8740037800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.17:49480/0 (socket says 172.21.15.17:49480) 2024-03-16T10:11:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7f8747fff640 1 -- 172.21.15.17:0/3573488562 learned_addr learned my addr 172.21.15.17:0/3573488562 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 624483950 0 0) 0x7f87400363a0 con 0x7f87400ea970 2024-03-16T10:11:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8738003510 con 0x7f87400ea970 2024-03-16T10:11:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4251592128 0 0) 0x7f8740035820 con 0x7f87400eadb0 2024-03-16T10:11:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87400363a0 con 0x7f87400eadb0 2024-03-16T10:11:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3022526208 0 0) 0x7f8738003510 con 0x7f87400ea970 2024-03-16T10:11:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8740035fc0 msgr2=0x7f8740037800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 --2- 172.21.15.17:0/3573488562 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8740035fc0 0x7f8740037800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 >> v1:172.21.15.174:6789/0 conn(0x7f87400eadb0 legacy=0x7f87400f4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8740039c90 con 0x7f87400ea970 2024-03-16T10:11:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f8734001680 con 0x7f87400ea970 2024-03-16T10:11:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 >> v1:172.21.15.17:6789/0 conn(0x7f87400ea970 legacy=0x7f87400e3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 --2- 172.21.15.17:0/3573488562 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8738003ce0 0x7f87380040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 --2- 172.21.15.17:0/3573488562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8738004620 0x7f873800d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 --2- 172.21.15.17:0/3573488562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f873800d960 0x7f873800fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f87400ea450 con 0x7f873800d960 2024-03-16T10:11:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8738010370 con 0x7f8738004620 2024-03-16T10:11:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8738010570 con 0x7f8738003ce0 2024-03-16T10:11:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f874cb2e640 1 --2- 172.21.15.17:0/3573488562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f873800d960 0x7f873800fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f87477fe640 1 --2- 172.21.15.17:0/3573488562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8738004620 0x7f873800d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f8747fff640 1 --2- 172.21.15.17:0/3573488562 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8738003ce0 0x7f87380040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f874cb2e640 1 -- 172.21.15.17:0/3573488562 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8738003ce0 msgr2=0x7f87380040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f874cb2e640 1 --2- 172.21.15.17:0/3573488562 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8738003ce0 0x7f87380040e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f874cb2e640 1 -- 172.21.15.17:0/3573488562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8738004620 msgr2=0x7f873800d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f874cb2e640 1 --2- 172.21.15.17:0/3573488562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8738004620 0x7f873800d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f874cb2e640 1 -- 172.21.15.17:0/3573488562 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8740039c90 con 0x7f873800d960 2024-03-16T10:11:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7fa60d7fa640 1 -- 172.21.15.17:0/781878902 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa5f801a070 con 0x7fa61008b2d0 2024-03-16T10:11:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.889+0000 7fa60d7fa640 1 --2- 172.21.15.17:0/781878902 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa5f0067730 0x7fa5f0069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7fa60d7fa640 1 -- 172.21.15.17:0/781878902 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa5f80852e0 con 0x7fa61008b2d0 2024-03-16T10:11:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7fa60d7fa640 1 --2- 172.21.15.17:0/781878902 >> [v2:172.21.15.17:6801/1622136967,v1:172.21.15.17:6804/1622136967] conn(0x7fa5f006f9b0 0x7fa5f0071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7fa60d7fa640 1 -- 172.21.15.17:0/781878902 --> [v2:172.21.15.17:6801/1622136967,v1:172.21.15.17:6804/1622136967] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa5f0072480 con 0x7fa5f006f9b0 2024-03-16T10:11:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7fa60d7fa640 1 -- 172.21.15.17:0/781878902 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa5f80856d0 con 0x7fa61008b2d0 2024-03-16T10:11:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7fa60ffff640 1 --2- 172.21.15.17:0/781878902 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa5f0067730 0x7fa5f0069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7fa6151c4640 1 --2- 172.21.15.17:0/781878902 >> [v2:172.21.15.17:6801/1622136967,v1:172.21.15.17:6804/1622136967] conn(0x7fa5f006f9b0 0x7fa5f0071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7fa60ffff640 1 --2- 172.21.15.17:0/781878902 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa5f0067730 0x7fa5f0069bf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6000b7000 tx=0x7fa600010040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7fa6151c4640 1 --2- 172.21.15.17:0/781878902 >> [v2:172.21.15.17:6801/1622136967,v1:172.21.15.17:6804/1622136967] conn(0x7fa5f006f9b0 0x7fa5f0071dd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.893+0000 7f874cb2e640 1 --2- 172.21.15.17:0/3573488562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f873800d960 0x7f873800fd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8748068950 tx=0x7f87480898f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cdd567f7b7a4e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7f8f697fa640 1 -- 172.21.15.17:0/2235953556 --> [v2:172.21.15.17:6824/3540530625,v1:172.21.15.17:6825/3540530625] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f5c014ec0 con 0x7f8f5806f860 2024-03-16T10:11:03.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874804fb20 con 0x7f873800d960 2024-03-16T10:11:03.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8748114730 con 0x7f873800d960 2024-03-16T10:11:03.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7f8746ffd640 1 -- 172.21.15.17:0/3573488562 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748114ba0 con 0x7f873800d960 2024-03-16T10:11:03.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7fa60d7fa640 1 -- 172.21.15.17:0/781878902 <== osd.2 v2:172.21.15.17:6801/1622136967 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa5f0072480 con 0x7fa5f006f9b0 2024-03-16T10:11:03.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7f874d32f640 1 -- 172.21.15.17:0/3573488562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f873800d960 msgr2=0x7f873800fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7f874d32f640 1 --2- 172.21.15.17:0/3573488562 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f873800d960 0x7f873800fd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8748068950 tx=0x7f87480898f0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7f874cb2e640 1 -- 172.21.15.17:0/3573488562 reap_dead start 2024-03-16T10:11:03.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7f874d32f640 1 -- 172.21.15.17:0/3573488562 shutdown_connections 2024-03-16T10:11:03.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7f874d32f640 1 -- 172.21.15.17:0/3573488562 >> 172.21.15.17:0/3573488562 conn(0x7f874003a860 msgr2=0x7f874003ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7f8f697fa640 1 -- 172.21.15.17:0/2235953556 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8f5c0892c0 con 0x7f8f6c0b5380 2024-03-16T10:11:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7f874d32f640 1 -- 172.21.15.17:0/3573488562 shutdown_connections 2024-03-16T10:11:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7f874d32f640 1 -- 172.21.15.17:0/3573488562 wait complete. 2024-03-16T10:11:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7f874d32f640 1 Processor -- start 2024-03-16T10:11:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.897+0000 7f874d32f640 1 -- start start 2024-03-16T10:11:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f874d32f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f873800d960 0x7f87400edce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f874d32f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87400e6750 0x7f87401802a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f874d32f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8740182fa0 0x7f8740183430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f874d32f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8740037470 con 0x7f873800d960 2024-03-16T10:11:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f874d32f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8740038390 con 0x7f87400e6750 2024-03-16T10:11:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f874d32f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f87400e6ed0 con 0x7f8740182fa0 2024-03-16T10:11:03.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f8747fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f873800d960 0x7f87400edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f8747fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f873800d960 0x7f87400edce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33734/0 (socket says 172.21.15.17:33734) 2024-03-16T10:11:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f8747fff640 1 -- 172.21.15.17:0/3686465002 learned_addr learned my addr 172.21.15.17:0/3686465002 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f874cb2e640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8740182fa0 0x7f8740183430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f87477fe640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87400e6750 0x7f87401802a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f8747fff640 1 -- 172.21.15.17:0/3686465002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8740182fa0 msgr2=0x7f8740183430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f8747fff640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8740182fa0 0x7f8740183430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f8747fff640 1 -- 172.21.15.17:0/3686465002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87400e6750 msgr2=0x7f87401802a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f8747fff640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87400e6750 0x7f87401802a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f8747fff640 1 -- 172.21.15.17:0/3686465002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87400ecec0 con 0x7f873800d960 2024-03-16T10:11:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f87477fe640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87400e6750 0x7f87401802a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f8747fff640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f873800d960 0x7f87400edce0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f873c010790 tx=0x7f873c010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f8744ff9640 1 -- 172.21.15.17:0/3686465002 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c01a020 con 0x7f873800d960 2024-03-16T10:11:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f874d32f640 1 -- 172.21.15.17:0/3686465002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8740034450 con 0x7f873800d960 2024-03-16T10:11:03.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f874d32f640 1 -- 172.21.15.17:0/3686465002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87400e6bf0 con 0x7f873800d960 2024-03-16T10:11:03.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f8744ff9640 1 -- 172.21.15.17:0/3686465002 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f873c00e8a0 con 0x7f873800d960 2024-03-16T10:11:03.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f8744ff9640 1 -- 172.21.15.17:0/3686465002 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f873c00ec70 con 0x7f873800d960 2024-03-16T10:11:03.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f8f697fa640 1 -- 172.21.15.17:0/2235953556 <== osd.0 v2:172.21.15.17:6824/3540530625 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f5c014ec0 con 0x7f8f5806f860 2024-03-16T10:11:03.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.901+0000 7f8744ff9640 1 -- 172.21.15.17:0/3686465002 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f873c01d020 con 0x7f873800d960 2024-03-16T10:11:03.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.905+0000 7f8744ff9640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f872c067660 0x7f872c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.905+0000 7f8744ff9640 1 -- 172.21.15.17:0/3686465002 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f873c085ca0 con 0x7f873800d960 2024-03-16T10:11:03.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.905+0000 7f87477fe640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f872c067660 0x7f872c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.905+0000 7f87477fe640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f872c067660 0x7f872c069b20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8734001ec0 tx=0x7f8734009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7ff0a37fe640 1 --2- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff08c067760 0x7ff08c069c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7f874d32f640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.17:6800/3058835082,v1:172.21.15.17:6802/3058835082] conn(0x7f8728001660 0x7f8728003b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7f874d32f640 1 -- 172.21.15.17:0/3686465002 --> [v2:172.21.15.17:6800/3058835082,v1:172.21.15.17:6802/3058835082] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8728006bc0 con 0x7f8728001660 2024-03-16T10:11:03.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7f874cb2e640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.17:6800/3058835082,v1:172.21.15.17:6802/3058835082] conn(0x7f8728001660 0x7f8728003b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7ff0a37fe640 1 -- 172.21.15.17:0/2605683738 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff0a4089470 con 0x7ff0ac148cc0 2024-03-16T10:11:03.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7ff0aa59c640 1 --2- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff08c067760 0x7ff08c069c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7ff0a37fe640 1 --2- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:6800/1721037852,v1:172.21.15.174:6801/1721037852] conn(0x7ff08c06f9e0 0x7ff08c071e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7f261eeed640 1 -- 172.21.15.17:0/3545818904 --> [v2:172.21.15.174:6819/326911164,v1:172.21.15.174:6822/326911164] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f25f4002d60 con 0x7f25fc06f860 2024-03-16T10:11:03.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7ff0a37fe640 1 -- 172.21.15.17:0/2605683738 --> [v2:172.21.15.174:6800/1721037852,v1:172.21.15.174:6801/1721037852] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0a4009ae0 con 0x7ff08c06f9e0 2024-03-16T10:11:03.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7ff0a37fe640 1 -- 172.21.15.17:0/2605683738 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff0a4090050 con 0x7ff0ac148cc0 2024-03-16T10:11:03.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7f874cb2e640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.17:6800/3058835082,v1:172.21.15.17:6802/3058835082] conn(0x7f8728001660 0x7f8728003b20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7ff0a1d9b640 1 --2- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:6800/1721037852,v1:172.21.15.174:6801/1721037852] conn(0x7ff08c06f9e0 0x7ff08c071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7f26157fa640 1 -- 172.21.15.17:0/3545818904 <== osd.4 v2:172.21.15.174:6819/326911164 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f25f4002d60 con 0x7f25fc06f860 2024-03-16T10:11:03.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7ff0aa59c640 1 --2- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff08c067760 0x7ff08c069c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0ad000 tx=0x7ff09c004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7ff0a1d9b640 1 --2- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:6800/1721037852,v1:172.21.15.174:6801/1721037852] conn(0x7ff08c06f9e0 0x7ff08c071e00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7f8744ff9640 1 -- 172.21.15.17:0/3686465002 <== osd.1 v2:172.21.15.17:6800/3058835082 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8728006bc0 con 0x7f8728001660 2024-03-16T10:11:03.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.909+0000 7ff0a37fe640 1 -- 172.21.15.17:0/2605683738 <== osd.5 v2:172.21.15.174:6800/1721037852 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff0a4009ae0 con 0x7ff08c06f9e0 2024-03-16T10:11:03.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f261eeed640 1 -- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:6819/326911164,v1:172.21.15.174:6822/326911164] conn(0x7f25fc06f860 msgr2=0x7f25fc071c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3685809552 --> [v2:172.21.15.174:6803/4101740318,v1:172.21.15.174:6805/4101740318] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83e806fb50 con 0x7f83d406f930 2024-03-16T10:11:03.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f261eeed640 1 --2- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:6819/326911164,v1:172.21.15.174:6822/326911164] conn(0x7f25fc06f860 0x7f25fc071c80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f261eeed640 1 -- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f25fc067660 msgr2=0x7f25fc069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f261eeed640 1 --2- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f25fc067660 0x7f25fc069b20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f26181494a0 tx=0x7f260c009190 comp rx=0 tx=0).stop 2024-03-16T10:11:03.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f261eeed640 1 -- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26180affc0 msgr2=0x7f26180abed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f83cf7fe640 1 -- 172.21.15.17:0/3685809552 <== osd.6 v2:172.21.15.174:6803/4101740318 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f83e806fb50 con 0x7f83d406f930 2024-03-16T10:11:03.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f261eeed640 1 --2- 172.21.15.17:0/3545818904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26180affc0 0x7f26180abed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f261001a3f0 tx=0x7f2610019560 comp rx=0 tx=0).stop 2024-03-16T10:11:03.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f261d48a640 1 -- 172.21.15.17:0/3545818904 reap_dead start 2024-03-16T10:11:03.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f261eeed640 1 -- 172.21.15.17:0/3545818904 shutdown_connections 2024-03-16T10:11:03.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f261eeed640 1 -- 172.21.15.17:0/3545818904 >> 172.21.15.17:0/3545818904 conn(0x7f26180b55c0 msgr2=0x7f26180b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f261eeed640 1 -- 172.21.15.17:0/3545818904 shutdown_connections 2024-03-16T10:11:03.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3685809552 >> [v2:172.21.15.174:6803/4101740318,v1:172.21.15.174:6805/4101740318] conn(0x7f83d406f930 msgr2=0x7f83d4071d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f83ec8fa640 1 --2- 172.21.15.17:0/3685809552 >> [v2:172.21.15.174:6803/4101740318,v1:172.21.15.174:6805/4101740318] conn(0x7f83d406f930 0x7f83d4071d50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f261eeed640 1 -- 172.21.15.17:0/3545818904 wait complete. 2024-03-16T10:11:03.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3685809552 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f83d40676b0 msgr2=0x7f83d4069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f83ec8fa640 1 --2- 172.21.15.17:0/3685809552 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f83d40676b0 0x7f83d4069b70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f83e80b4420 tx=0x7f83d8009190 comp rx=0 tx=0).stop 2024-03-16T10:11:03.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3685809552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b04f0 msgr2=0x7f83e81592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f83ec8fa640 1 --2- 172.21.15.17:0/3685809552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f83e80b04f0 0x7f83e81592e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f83e0008a50 tx=0x7f83e000e4f0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f83e759e640 1 -- 172.21.15.17:0/3685809552 reap_dead start 2024-03-16T10:11:03.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3685809552 shutdown_connections 2024-03-16T10:11:03.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3685809552 >> 172.21.15.17:0/3685809552 conn(0x7f83e806b290 msgr2=0x7f83e80afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3685809552 shutdown_connections 2024-03-16T10:11:03.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.917+0000 7f83ec8fa640 1 -- 172.21.15.17:0/3685809552 wait complete. 2024-03-16T10:11:03.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.921+0000 7f0ab8b97640 1 -- 172.21.15.17:0/2171641251 --> [v2:172.21.15.17:6803/1266087291,v1:172.21.15.17:6806/1266087291] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0ab40b5200 con 0x7f0a9c06fa70 2024-03-16T10:11:03.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.921+0000 7f0aa2ffd640 1 -- 172.21.15.17:0/2171641251 <== osd.3 v2:172.21.15.17:6803/1266087291 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0ab40b5200 con 0x7f0a9c06fa70 2024-03-16T10:11:03.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.929+0000 7f0aa0ff9640 1 -- 172.21.15.17:0/2171641251 >> [v2:172.21.15.17:6803/1266087291,v1:172.21.15.17:6806/1266087291] conn(0x7f0a9c06fa70 msgr2=0x7f0a9c071e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.929+0000 7f0aa0ff9640 1 --2- 172.21.15.17:0/2171641251 >> [v2:172.21.15.17:6803/1266087291,v1:172.21.15.17:6806/1266087291] conn(0x7f0a9c06fa70 0x7f0a9c071e90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.929+0000 7f0aa0ff9640 1 -- 172.21.15.17:0/2171641251 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0a9c067870 msgr2=0x7f0a9c069d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.929+0000 7f0aa0ff9640 1 --2- 172.21.15.17:0/2171641251 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0a9c067870 0x7f0a9c069d30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80ad000 tx=0x7f0aa80a6000 comp rx=0 tx=0).stop 2024-03-16T10:11:03.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.929+0000 7f0aa0ff9640 1 -- 172.21.15.17:0/2171641251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ab40b60e0 msgr2=0x7f0ab40b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.929+0000 7f0aa0ff9640 1 --2- 172.21.15.17:0/2171641251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ab40b60e0 0x7f0ab40b31f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400c400 tx=0x7f0aa400c8c0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.929+0000 7f0ab2d9d640 1 -- 172.21.15.17:0/2171641251 reap_dead start 2024-03-16T10:11:03.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.929+0000 7f0aa0ff9640 1 -- 172.21.15.17:0/2171641251 shutdown_connections 2024-03-16T10:11:03.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.929+0000 7f0aa0ff9640 1 -- 172.21.15.17:0/2171641251 >> 172.21.15.17:0/2171641251 conn(0x7f0ab40b9a30 msgr2=0x7f0ab4147790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.929+0000 7f0aa0ff9640 1 -- 172.21.15.17:0/2171641251 shutdown_connections 2024-03-16T10:11:03.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.929+0000 7f0aa0ff9640 1 -- 172.21.15.17:0/2171641251 wait complete. 2024-03-16T10:11:03.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.933+0000 7f8f72b5f640 1 -- 172.21.15.17:0/2235953556 --> [v2:172.21.15.17:6824/3540530625,v1:172.21.15.17:6825/3540530625] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8f6c068360 con 0x7f8f5806f860 2024-03-16T10:11:03.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.933+0000 7f8f697fa640 1 -- 172.21.15.17:0/2235953556 <== osd.0 v2:172.21.15.17:6824/3540530625 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8f6c068360 con 0x7f8f5806f860 2024-03-16T10:11:03.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7f8f72b5f640 1 -- 172.21.15.17:0/2235953556 >> [v2:172.21.15.17:6824/3540530625,v1:172.21.15.17:6825/3540530625] conn(0x7f8f5806f860 msgr2=0x7f8f58071c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7f8f72b5f640 1 --2- 172.21.15.17:0/2235953556 >> [v2:172.21.15.17:6824/3540530625,v1:172.21.15.17:6825/3540530625] conn(0x7f8f5806f860 0x7f8f58071c80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7f8f72b5f640 1 -- 172.21.15.17:0/2235953556 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8f58067660 msgr2=0x7f8f58069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7f8f72b5f640 1 --2- 172.21.15.17:0/2235953556 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8f58067660 0x7f8f58069b20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8f640af000 tx=0x7f8f640a7000 comp rx=0 tx=0).stop 2024-03-16T10:11:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7f8f72b5f640 1 -- 172.21.15.17:0/2235953556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c0b5380 msgr2=0x7f8f6c1ec5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7f8f72b5f640 1 --2- 172.21.15.17:0/2235953556 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c0b5380 0x7f8f6c1ec5c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c001390 tx=0x7f8f5c00cbe0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7f8f710fc640 1 -- 172.21.15.17:0/2235953556 reap_dead start 2024-03-16T10:11:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7f8f72b5f640 1 -- 172.21.15.17:0/2235953556 shutdown_connections 2024-03-16T10:11:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7f8f72b5f640 1 -- 172.21.15.17:0/2235953556 >> 172.21.15.17:0/2235953556 conn(0x7f8f6c06b460 msgr2=0x7f8f6c0b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7f8f72b5f640 1 -- 172.21.15.17:0/2235953556 shutdown_connections 2024-03-16T10:11:03.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7fa6159c5640 1 -- 172.21.15.17:0/781878902 --> [v2:172.21.15.17:6801/1622136967,v1:172.21.15.17:6804/1622136967] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa610157820 con 0x7fa5f006f9b0 2024-03-16T10:11:03.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7fa60d7fa640 1 -- 172.21.15.17:0/781878902 <== osd.2 v2:172.21.15.17:6801/1622136967 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa610157820 con 0x7fa5f006f9b0 2024-03-16T10:11:03.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7fa6159c5640 1 -- 172.21.15.17:0/781878902 >> [v2:172.21.15.17:6801/1622136967,v1:172.21.15.17:6804/1622136967] conn(0x7fa5f006f9b0 msgr2=0x7fa5f0071dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7fa6159c5640 1 --2- 172.21.15.17:0/781878902 >> [v2:172.21.15.17:6801/1622136967,v1:172.21.15.17:6804/1622136967] conn(0x7fa5f006f9b0 0x7fa5f0071dd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7fa6159c5640 1 -- 172.21.15.17:0/781878902 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa5f0067730 msgr2=0x7fa5f0069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7fa6159c5640 1 --2- 172.21.15.17:0/781878902 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa5f0067730 0x7fa5f0069bf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa6000b7000 tx=0x7fa600010040 comp rx=0 tx=0).stop 2024-03-16T10:11:03.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7fa6159c5640 1 -- 172.21.15.17:0/781878902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa61008b2d0 msgr2=0x7fa6100b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7fa6159c5640 1 --2- 172.21.15.17:0/781878902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa61008b2d0 0x7fa6100b1290 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800c4f0 tx=0x7fa5f800c9b0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7fa6151c4640 1 -- 172.21.15.17:0/781878902 reap_dead start 2024-03-16T10:11:03.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7fa6159c5640 1 -- 172.21.15.17:0/781878902 shutdown_connections 2024-03-16T10:11:03.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7fa6159c5640 1 -- 172.21.15.17:0/781878902 >> 172.21.15.17:0/781878902 conn(0x7fa6100b5550 msgr2=0x7fa610155fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7fa6159c5640 1 -- 172.21.15.17:0/781878902 shutdown_connections 2024-03-16T10:11:03.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7fa6159c5640 1 -- 172.21.15.17:0/781878902 wait complete. 2024-03-16T10:11:03.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.937+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/2605683738 --> [v2:172.21.15.174:6800/1721037852,v1:172.21.15.174:6801/1721037852] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff0ac0b44d0 con 0x7ff08c06f9e0 2024-03-16T10:11:03.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7f8f72b5f640 1 -- 172.21.15.17:0/2235953556 wait complete. 2024-03-16T10:11:03.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd238fca640 1 Processor -- start 2024-03-16T10:11:03.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7ff0a37fe640 1 -- 172.21.15.17:0/2605683738 <== osd.5 v2:172.21.15.174:6800/1721037852 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff0ac0b44d0 con 0x7ff08c06f9e0 2024-03-16T10:11:03.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd238fca640 1 -- start start 2024-03-16T10:11:03.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd238fca640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd22c036aa0 0x7fd22c0d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd238fca640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd22c0cab10 0x7fd22c0d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd238fca640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd22c036e80 con 0x7fd22c0d10b0 2024-03-16T10:11:03.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd238fca640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd22c038390 con 0x7fd22c0cab10 2024-03-16T10:11:03.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd238fca640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd22c0d4960 con 0x7fd22c036aa0 2024-03-16T10:11:03.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd233fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd22c0cab10 0x7fd22c0d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd233fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd22c0cab10 0x7fd22c0d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33738/0 (socket says 172.21.15.17:33738) 2024-03-16T10:11:03.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd233fff640 1 -- 172.21.15.17:0/1888255203 learned_addr learned my addr 172.21.15.17:0/1888255203 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:03.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd2337fe640 1 --2- 172.21.15.17:0/1888255203 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd22c036aa0 0x7fd22c0d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd233fff640 1 -- 172.21.15.17:0/1888255203 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd22c036aa0 msgr2=0x7fd22c0d4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:03.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd233fff640 1 --2- 172.21.15.17:0/1888255203 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd22c036aa0 0x7fd22c0d4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd233fff640 1 -- 172.21.15.17:0/1888255203 >> v1:172.21.15.17:6789/0 conn(0x7fd22c0d10b0 legacy=0x7fd22c0ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.941+0000 7fd233fff640 1 -- 172.21.15.17:0/1888255203 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c029bb0 con 0x7fd22c0cab10 2024-03-16T10:11:03.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:6800/1721037852,v1:172.21.15.174:6801/1721037852] conn(0x7ff08c06f9e0 msgr2=0x7ff08c071e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7fd233fff640 1 --2- 172.21.15.17:0/1888255203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd22c0cab10 0x7fd22c0d1510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd220009870 tx=0x7fd22001c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238c7ea0ab6963a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7ff0b0ef0640 1 --2- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:6800/1721037852,v1:172.21.15.174:6801/1721037852] conn(0x7ff08c06f9e0 0x7ff08c071e00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7fd2327fc640 1 -- 172.21.15.17:0/1888255203 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2200b2070 con 0x7fd22c0cab10 2024-03-16T10:11:03.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff08c067760 msgr2=0x7ff08c069c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7ff0b0ef0640 1 --2- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff08c067760 0x7ff08c069c20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0ad000 tx=0x7ff09c004740 comp rx=0 tx=0).stop 2024-03-16T10:11:03.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0ac148cc0 msgr2=0x7ff0ac1356a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7ff0b0ef0640 1 --2- 172.21.15.17:0/2605683738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0ac148cc0 0x7ff0ac1356a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400ed40 tx=0x7ff0a400ce90 comp rx=0 tx=0).stop 2024-03-16T10:11:03.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7ff0aad9d640 1 -- 172.21.15.17:0/2605683738 reap_dead start 2024-03-16T10:11:03.951 INFO:teuthology.orchestra.run.smithi017.stdout:55834574853 2024-03-16T10:11:03.951 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-16T10:11:03.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7f874d32f640 1 -- 172.21.15.17:0/3686465002 --> [v2:172.21.15.17:6800/3058835082,v1:172.21.15.17:6802/3058835082] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8728005c50 con 0x7f8728001660 2024-03-16T10:11:03.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7fd2327fc640 1 -- 172.21.15.17:0/1888255203 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2200ad680 con 0x7fd22c0cab10 2024-03-16T10:11:03.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7fd2327fc640 1 -- 172.21.15.17:0/1888255203 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2200ada50 con 0x7fd22c0cab10 2024-03-16T10:11:03.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7fd238fca640 1 -- 172.21.15.17:0/1888255203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd22c0cab10 msgr2=0x7fd22c0d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7fd238fca640 1 --2- 172.21.15.17:0/1888255203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd22c0cab10 0x7fd22c0d1510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd220009870 tx=0x7fd22001c580 comp rx=0 tx=0).stop 2024-03-16T10:11:03.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7f8744ff9640 1 -- 172.21.15.17:0/3686465002 <== osd.1 v2:172.21.15.17:6800/3058835082 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8728005c50 con 0x7f8728001660 2024-03-16T10:11:03.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7f874d32f640 1 -- 172.21.15.17:0/3686465002 >> [v2:172.21.15.17:6800/3058835082,v1:172.21.15.17:6802/3058835082] conn(0x7f8728001660 msgr2=0x7f8728003b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7f874d32f640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.17:6800/3058835082,v1:172.21.15.17:6802/3058835082] conn(0x7f8728001660 0x7f8728003b20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7fd238fca640 1 -- 172.21.15.17:0/1888255203 shutdown_connections 2024-03-16T10:11:03.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7fd238fca640 1 --2- 172.21.15.17:0/1888255203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd22c0cab10 0x7fd22c0d1510 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7fd238fca640 1 --2- 172.21.15.17:0/1888255203 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd22c036aa0 0x7fd22c0d4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7fd238fca640 1 -- 172.21.15.17:0/1888255203 >> 172.21.15.17:0/1888255203 conn(0x7fd22c03abd0 msgr2=0x7fd22c005ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7f874d32f640 1 -- 172.21.15.17:0/3686465002 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f872c067660 msgr2=0x7f872c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.945+0000 7f874d32f640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f872c067660 0x7f872c069b20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8734001ec0 tx=0x7f8734009040 comp rx=0 tx=0).stop 2024-03-16T10:11:03.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.949+0000 7f874d32f640 1 -- 172.21.15.17:0/3686465002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f873800d960 msgr2=0x7f87400edce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.949+0000 7f874d32f640 1 --2- 172.21.15.17:0/3686465002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f873800d960 0x7f87400edce0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f873c010790 tx=0x7f873c010c50 comp rx=0 tx=0).stop 2024-03-16T10:11:03.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.949+0000 7f874cb2e640 1 -- 172.21.15.17:0/3686465002 reap_dead start 2024-03-16T10:11:03.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.949+0000 7f874d32f640 1 -- 172.21.15.17:0/3686465002 shutdown_connections 2024-03-16T10:11:03.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.949+0000 7f874d32f640 1 -- 172.21.15.17:0/3686465002 >> 172.21.15.17:0/3686465002 conn(0x7f874003a860 msgr2=0x7f87400e4170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.949+0000 7f874d32f640 1 -- 172.21.15.17:0/3686465002 shutdown_connections 2024-03-16T10:11:03.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.949+0000 7f874d32f640 1 -- 172.21.15.17:0/3686465002 wait complete. 2024-03-16T10:11:03.956 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-16T10:11:03.956 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-16T10:11:03.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.953+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/2605683738 shutdown_connections 2024-03-16T10:11:03.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.953+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/2605683738 >> 172.21.15.17:0/2605683738 conn(0x7ff0ac06b460 msgr2=0x7ff0ac1495e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:03.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.953+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/2605683738 shutdown_connections 2024-03-16T10:11:03.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.953+0000 7fd238fca640 1 -- 172.21.15.17:0/1888255203 shutdown_connections 2024-03-16T10:11:03.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.953+0000 7ff0b0ef0640 1 -- 172.21.15.17:0/2605683738 wait complete. 2024-03-16T10:11:03.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.953+0000 7fd238fca640 1 -- 172.21.15.17:0/1888255203 wait complete. 2024-03-16T10:11:03.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.953+0000 7fd238fca640 1 Processor -- start 2024-03-16T10:11:03.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.953+0000 7fd238fca640 1 -- start start 2024-03-16T10:11:03.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.953+0000 7fd238fca640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22c036aa0 0x7fd22c0cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.953+0000 7fd238fca640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd22c0cab10 0x7fd22c0bdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.953+0000 7fd238fca640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd22c0d10b0 0x7fd22c0be210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.953+0000 7fd238fca640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c0dbc90 con 0x7fd22c0cab10 2024-03-16T10:11:03.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.953+0000 7fd238fca640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c0dbb10 con 0x7fd22c036aa0 2024-03-16T10:11:03.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.953+0000 7fd238fca640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd22c0dbe10 con 0x7fd22c0d10b0 2024-03-16T10:11:03.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.957+0000 7fd232ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd22c0cab10 0x7fd22c0bdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.957+0000 7fd232ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd22c0cab10 0x7fd22c0bdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33752/0 (socket says 172.21.15.17:33752) 2024-03-16T10:11:03.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.957+0000 7fd232ffd640 1 -- 172.21.15.17:0/3254792663 learned_addr learned my addr 172.21.15.17:0/3254792663 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:03.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.957+0000 7fd233fff640 1 --2- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd22c0d10b0 0x7fd22c0be210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.957+0000 7fd232ffd640 1 -- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd22c0d10b0 msgr2=0x7fd22c0be210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:03.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.957+0000 7fd232ffd640 1 --2- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd22c0d10b0 0x7fd22c0be210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.957+0000 7fd232ffd640 1 -- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22c036aa0 msgr2=0x7fd22c0cb480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T10:11:03.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.957+0000 7fd232ffd640 1 --2- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22c036aa0 0x7fd22c0cb480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:03.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.957+0000 7fd232ffd640 1 -- 172.21.15.17:0/3254792663 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c0dae30 con 0x7fd22c0cab10 2024-03-16T10:11:03.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.957+0000 7fd232ffd640 1 --2- 172.21.15.17:0/3254792663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd22c0cab10 0x7fd22c0bdcd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd23402d750 tx=0x7fd234068d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.968 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-16T10:11:03.968 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-16T10:11:03.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.965+0000 7fd21ffff640 1 -- 172.21.15.17:0/3254792663 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234071b00 con 0x7fd22c0cab10 2024-03-16T10:11:03.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.965+0000 7fd21ffff640 1 -- 172.21.15.17:0/3254792663 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd234071d40 con 0x7fd22c0cab10 2024-03-16T10:11:03.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.965+0000 7fd21ffff640 1 -- 172.21.15.17:0/3254792663 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234072660 con 0x7fd22c0cab10 2024-03-16T10:11:03.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.965+0000 7fd238fca640 1 -- 172.21.15.17:0/3254792663 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22c0bd810 con 0x7fd22c0cab10 2024-03-16T10:11:03.971 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-16T10:11:03.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.965+0000 7fd238fca640 1 -- 172.21.15.17:0/3254792663 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd22c0cdfc0 con 0x7fd22c0cab10 2024-03-16T10:11:03.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.965+0000 7fd21ffff640 1 -- 172.21.15.17:0/3254792663 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd234072800 con 0x7fd22c0cab10 2024-03-16T10:11:03.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.969+0000 7fd238fca640 1 -- 172.21.15.17:0/3254792663 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd22c0bf1d0 con 0x7fd22c0cab10 2024-03-16T10:11:03.972 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-16T10:11:03.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.969+0000 7fd21ffff640 1 --2- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd218067730 0x7fd218069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.969+0000 7fd2337fe640 1 --2- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd218067730 0x7fd218069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.969+0000 7fd21ffff640 1 -- 172.21.15.17:0/3254792663 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd234100e80 con 0x7fd22c0cab10 2024-03-16T10:11:03.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.969+0000 7fd21ffff640 1 --2- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:6816/1083843796,v1:172.21.15.174:6817/1083843796] conn(0x7fd21806f930 0x7fd218071d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:03.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.969+0000 7fd21ffff640 1 -- 172.21.15.17:0/3254792663 --> [v2:172.21.15.174:6816/1083843796,v1:172.21.15.174:6817/1083843796] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd218072400 con 0x7fd21806f930 2024-03-16T10:11:03.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.969+0000 7fd21ffff640 1 -- 172.21.15.17:0/3254792663 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd2340cd1c0 con 0x7fd22c0cab10 2024-03-16T10:11:03.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.969+0000 7fd233fff640 1 --2- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:6816/1083843796,v1:172.21.15.174:6817/1083843796] conn(0x7fd21806f930 0x7fd218071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:03.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.969+0000 7fd233fff640 1 --2- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:6816/1083843796,v1:172.21.15.174:6817/1083843796] conn(0x7fd21806f930 0x7fd218071d50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.969+0000 7fd2337fe640 1 --2- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd218067730 0x7fd218069bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd22c03d830 tx=0x7fd228008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:03.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.973+0000 7fd21ffff640 1 -- 172.21.15.17:0/3254792663 <== osd.7 v2:172.21.15.174:6816/1083843796 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd218072400 con 0x7fd21806f930 2024-03-16T10:11:03.987 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-16T10:11:03.987 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-16T10:11:03.988 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-16T10:11:03.988 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-16T10:11:04.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:03.997+0000 7fd238fca640 1 -- 172.21.15.17:0/3254792663 --> [v2:172.21.15.174:6816/1083843796,v1:172.21.15.174:6817/1083843796] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd22c0053a0 con 0x7fd21806f930 2024-03-16T10:11:04.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.001+0000 7fd21ffff640 1 -- 172.21.15.17:0/3254792663 <== osd.7 v2:172.21.15.174:6816/1083843796 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd22c0053a0 con 0x7fd21806f930 2024-03-16T10:11:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.001+0000 7fd238fca640 1 -- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:6816/1083843796,v1:172.21.15.174:6817/1083843796] conn(0x7fd21806f930 msgr2=0x7fd218071d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.001+0000 7fd238fca640 1 --2- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:6816/1083843796,v1:172.21.15.174:6817/1083843796] conn(0x7fd21806f930 0x7fd218071d50 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.001+0000 7fd238fca640 1 -- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd218067730 msgr2=0x7fd218069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.001+0000 7fd238fca640 1 --2- 172.21.15.17:0/3254792663 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd218067730 0x7fd218069bf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd22c03d830 tx=0x7fd228008040 comp rx=0 tx=0).stop 2024-03-16T10:11:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.001+0000 7fd238fca640 1 -- 172.21.15.17:0/3254792663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd22c0cab10 msgr2=0x7fd22c0bdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.001+0000 7fd238fca640 1 --2- 172.21.15.17:0/3254792663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd22c0cab10 0x7fd22c0bdcd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd23402d750 tx=0x7fd234068d80 comp rx=0 tx=0).stop 2024-03-16T10:11:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.001+0000 7fd233fff640 1 -- 172.21.15.17:0/3254792663 reap_dead start 2024-03-16T10:11:04.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.001+0000 7fd238fca640 1 -- 172.21.15.17:0/3254792663 shutdown_connections 2024-03-16T10:11:04.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.001+0000 7fd238fca640 1 -- 172.21.15.17:0/3254792663 >> 172.21.15.17:0/3254792663 conn(0x7fd22c03abd0 msgr2=0x7fd22c0c9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.001+0000 7fd238fca640 1 -- 172.21.15.17:0/3254792663 shutdown_connections 2024-03-16T10:11:04.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.001+0000 7fd238fca640 1 -- 172.21.15.17:0/3254792663 wait complete. 2024-03-16T10:11:04.012 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-16T10:11:04.012 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-16T10:11:04.030 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-16T10:11:04.030 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-16T10:11:04.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.117+0000 7ffa79a6f640 1 Processor -- start 2024-03-16T10:11:04.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.117+0000 7ffa79a6f640 1 -- start start 2024-03-16T10:11:04.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.121+0000 7ffa79a6f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa740b2a70 0x7ffa740b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.121+0000 7ffa79a6f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ffa740b0e70 0x7ffa7414da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.121+0000 7ffa79a6f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa74147610 con 0x7ffa74143d70 2024-03-16T10:11:04.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.121+0000 7ffa79a6f640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ffa74144150 con 0x7ffa740b0e70 2024-03-16T10:11:04.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.121+0000 7ffa79a6f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ffa74146a70 con 0x7ffa740b2a70 2024-03-16T10:11:04.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.121+0000 7ffa78a6d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ffa74143d70 0x7ffa74146f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56072/0 (socket says 172.21.15.17:56072) 2024-03-16T10:11:04.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.121+0000 7ffa78a6d640 1 -- 172.21.15.17:0/1046452298 learned_addr learned my addr 172.21.15.17:0/1046452298 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:04.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.121+0000 7ffa73fff640 1 --2- 172.21.15.17:0/1046452298 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa740b2a70 0x7ffa740b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.121+0000 7ffa73fff640 1 -- 172.21.15.17:0/1046452298 >> v2:172.21.15.174:3300/0 conn(0x7ffa740b0e70 msgr2=0x7ffa7414da30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-16T10:11:04.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.121+0000 7ffa73fff640 1 --2- 172.21.15.17:0/1046452298 >> v2:172.21.15.174:3300/0 conn(0x7ffa740b0e70 0x7ffa7414da30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.121+0000 7ffa73fff640 1 -- 172.21.15.17:0/1046452298 >> v1:172.21.15.17:6789/0 conn(0x7ffa74143d70 legacy=0x7ffa74146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.121+0000 7ffa73fff640 1 -- 172.21.15.17:0/1046452298 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa740b5d00 con 0x7ffa740b2a70 2024-03-16T10:11:04.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.125+0000 7ffa73fff640 1 --2- 172.21.15.17:0/1046452298 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa740b2a70 0x7ffa740b56e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00aea0 tx=0x7ffa6c01d920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d527eaf30add38e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa737fe640 1 -- 172.21.15.17:0/1046452298 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0a6560 con 0x7ffa740b2a70 2024-03-16T10:11:04.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa737fe640 1 -- 172.21.15.17:0/1046452298 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa6c0a6700 con 0x7ffa740b2a70 2024-03-16T10:11:04.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa737fe640 1 -- 172.21.15.17:0/1046452298 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6c0a6ad0 con 0x7ffa740b2a70 2024-03-16T10:11:04.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa79a6f640 1 -- 172.21.15.17:0/1046452298 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa740b2a70 msgr2=0x7ffa740b56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa79a6f640 1 --2- 172.21.15.17:0/1046452298 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa740b2a70 0x7ffa740b56e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c00aea0 tx=0x7ffa6c01d920 comp rx=0 tx=0).stop 2024-03-16T10:11:04.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa79a6f640 1 -- 172.21.15.17:0/1046452298 shutdown_connections 2024-03-16T10:11:04.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa79a6f640 1 --2- 172.21.15.17:0/1046452298 >> v2:172.21.15.174:3300/0 conn(0x7ffa740b0e70 0x7ffa7414da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa79a6f640 1 --2- 172.21.15.17:0/1046452298 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa740b2a70 0x7ffa740b56e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa79a6f640 1 -- 172.21.15.17:0/1046452298 >> 172.21.15.17:0/1046452298 conn(0x7ffa7406b290 msgr2=0x7ffa7406b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa79a6f640 1 -- 172.21.15.17:0/1046452298 shutdown_connections 2024-03-16T10:11:04.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa79a6f640 1 -- 172.21.15.17:0/1046452298 wait complete. 2024-03-16T10:11:04.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa79a6f640 1 Processor -- start 2024-03-16T10:11:04.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa79a6f640 1 -- start start 2024-03-16T10:11:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa79a6f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa740b0e70 0x7ffa740a95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.133+0000 7ffa79a6f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa740b2a70 0x7ffa741e1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa79a6f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa74143d70 0x7ffa741e1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa79a6f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffa74150910 con 0x7ffa740b0e70 2024-03-16T10:11:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa79a6f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa74150790 con 0x7ffa740b2a70 2024-03-16T10:11:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa79a6f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ffa74150a90 con 0x7ffa74143d70 2024-03-16T10:11:04.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa78a6d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa740b0e70 0x7ffa740a95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa78a6d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa740b0e70 0x7ffa740a95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33756/0 (socket says 172.21.15.17:33756) 2024-03-16T10:11:04.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa78a6d640 1 -- 172.21.15.17:0/2225689859 learned_addr learned my addr 172.21.15.17:0/2225689859 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa73fff640 1 --2- 172.21.15.17:0/2225689859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa740b2a70 0x7ffa741e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa7926e640 1 --2- 172.21.15.17:0/2225689859 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa74143d70 0x7ffa741e1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa78a6d640 1 -- 172.21.15.17:0/2225689859 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa74143d70 msgr2=0x7ffa741e1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa78a6d640 1 --2- 172.21.15.17:0/2225689859 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa74143d70 0x7ffa741e1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa78a6d640 1 -- 172.21.15.17:0/2225689859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa740b2a70 msgr2=0x7ffa741e1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa78a6d640 1 --2- 172.21.15.17:0/2225689859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa740b2a70 0x7ffa741e1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa78a6d640 1 -- 172.21.15.17:0/2225689859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa740aaf50 con 0x7ffa740b0e70 2024-03-16T10:11:04.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa78a6d640 1 --2- 172.21.15.17:0/2225689859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa740b0e70 0x7ffa740a95d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400cc10 tx=0x7ffa6400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa717fa640 1 -- 172.21.15.17:0/2225689859 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa64018070 con 0x7ffa740b0e70 2024-03-16T10:11:04.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa717fa640 1 -- 172.21.15.17:0/2225689859 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa64004de0 con 0x7ffa740b0e70 2024-03-16T10:11:04.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa717fa640 1 -- 172.21.15.17:0/2225689859 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6400ad80 con 0x7ffa740b0e70 2024-03-16T10:11:04.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa79a6f640 1 -- 172.21.15.17:0/2225689859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa740abe80 con 0x7ffa740b0e70 2024-03-16T10:11:04.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.137+0000 7ffa79a6f640 1 -- 172.21.15.17:0/2225689859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa740ac440 con 0x7ffa740b0e70 2024-03-16T10:11:04.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.141+0000 7ffa79a6f640 1 -- 172.21.15.17:0/2225689859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa540052a0 con 0x7ffa740b0e70 2024-03-16T10:11:04.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.141+0000 7ffa717fa640 1 -- 172.21.15.17:0/2225689859 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7ffa6401b070 con 0x7ffa740b0e70 2024-03-16T10:11:04.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.141+0000 7ffa717fa640 1 --2- 172.21.15.17:0/2225689859 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ffa60067730 0x7ffa60069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.141+0000 7ffa717fa640 1 -- 172.21.15.17:0/2225689859 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ffa64085230 con 0x7ffa740b0e70 2024-03-16T10:11:04.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.141+0000 7ffa73fff640 1 --2- 172.21.15.17:0/2225689859 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ffa60067730 0x7ffa60069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.141+0000 7ff5ddb1a640 1 Processor -- start 2024-03-16T10:11:04.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.141+0000 7ff5ddb1a640 1 -- start start 2024-03-16T10:11:04.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.141+0000 7ffa73fff640 1 --2- 172.21.15.17:0/2225689859 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ffa60067730 0x7ffa60069bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0af000 tx=0x7ffa6c0a8070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7ff5ddb1a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff5d814aff0 0x7ff5d80727f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7ff5ddb1a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff5d8072d30 0x7ff5d8151a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7ff5ddb1a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5d80afaa0 con 0x7ff5d80b5810 2024-03-16T10:11:04.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7ff5ddb1a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff5d80aeb70 con 0x7ff5d8072d30 2024-03-16T10:11:04.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7ff5ddb1a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ff5d80b5bf0 con 0x7ff5d814aff0 2024-03-16T10:11:04.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95e52640 1 Processor -- start 2024-03-16T10:11:04.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95e52640 1 -- start start 2024-03-16T10:11:04.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7ff5dcb18640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff5d80b5810 0x7ff5d814a8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56078/0 (socket says 172.21.15.17:56078) 2024-03-16T10:11:04.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7ff5dcb18640 1 -- 172.21.15.17:0/1413869499 learned_addr learned my addr 172.21.15.17:0/1413869499 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:04.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7ff5dd319640 1 --2- 172.21.15.17:0/1413869499 >> v2:172.21.15.17:3300/0 conn(0x7ff5d8072d30 0x7ff5d8151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7ff5d7fff640 1 --2- 172.21.15.17:0/1413869499 >> v2:172.21.15.174:3300/0 conn(0x7ff5d814aff0 0x7ff5d80727f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95e52640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b90143d70 0x7f9b900b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95e52640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9b900b2a70 0x7f9b90146f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95e52640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9b900b0e70 0x7f9b900afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95e52640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9b90144150 con 0x7f9b900b0e70 2024-03-16T10:11:04.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95e52640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f9b90146a70 con 0x7f9b900b2a70 2024-03-16T10:11:04.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95e52640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9b900b5d00 con 0x7f9b90143d70 2024-03-16T10:11:04.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7ff5dd319640 1 -- 172.21.15.17:0/1413869499 >> v2:172.21.15.174:3300/0 conn(0x7ff5d814aff0 msgr2=0x7ff5d80727f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:04.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7ff5dd319640 1 --2- 172.21.15.17:0/1413869499 >> v2:172.21.15.174:3300/0 conn(0x7ff5d814aff0 0x7ff5d80727f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7ff5dd319640 1 -- 172.21.15.17:0/1413869499 >> v1:172.21.15.17:6789/0 conn(0x7ff5d80b5810 legacy=0x7ff5d814a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95651640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9b900b0e70 0x7f9b900afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95651640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9b900b0e70 0x7f9b900afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33772/0 (socket says 172.21.15.17:33772) 2024-03-16T10:11:04.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95651640 1 -- 172.21.15.17:0/1770115158 learned_addr learned my addr 172.21.15.17:0/1770115158 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b94e50640 1 --2- 172.21.15.17:0/1770115158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b90143d70 0x7f9b900b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b8ffff640 1 --2- 172.21.15.17:0/1770115158 >> v2:172.21.15.174:3300/0 conn(0x7f9b900b2a70 0x7f9b90146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95651640 1 -- 172.21.15.17:0/1770115158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b90143d70 msgr2=0x7f9b900b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:04.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95651640 1 --2- 172.21.15.17:0/1770115158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b90143d70 0x7f9b900b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95651640 1 -- 172.21.15.17:0/1770115158 >> v2:172.21.15.174:3300/0 conn(0x7f9b900b2a70 msgr2=0x7f9b90146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:04.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7ff5dd319640 1 -- 172.21.15.17:0/1413869499 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d8073160 con 0x7ff5d8072d30 2024-03-16T10:11:04.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95651640 1 --2- 172.21.15.17:0/1770115158 >> v2:172.21.15.174:3300/0 conn(0x7f9b900b2a70 0x7f9b90146f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.145+0000 7f9b95651640 1 -- 172.21.15.17:0/1770115158 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b90146700 con 0x7f9b900b0e70 2024-03-16T10:11:04.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.149+0000 7f9b95651640 1 --2- 172.21.15.17:0/1770115158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b900b0e70 0x7f9b900afd80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9b900b87e0 tx=0x7f9b8401c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcc5929fab49f15 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.149+0000 7ff5dd319640 1 --2- 172.21.15.17:0/1413869499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d8072d30 0x7ff5d8151a30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80af940 tx=0x7ff5d001cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6fbe439cad3d3c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.149+0000 7f9b8f7fe640 1 -- 172.21.15.17:0/1770115158 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b840ac070 con 0x7f9b900b0e70 2024-03-16T10:11:04.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.149+0000 7ffa717fa640 1 -- 172.21.15.17:0/2225689859 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ffa74147610 con 0x7ffa740b0e70 2024-03-16T10:11:04.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.149+0000 7ff5d77fe640 1 -- 172.21.15.17:0/1413869499 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00b2070 con 0x7ff5d8072d30 2024-03-16T10:11:04.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.153+0000 7f9b8f7fe640 1 -- 172.21.15.17:0/1770115158 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b840a6630 con 0x7f9b900b0e70 2024-03-16T10:11:04.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.153+0000 7f9b95e52640 1 -- 172.21.15.17:0/1770115158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b900b0e70 msgr2=0x7f9b900afd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.153+0000 7f9b95e52640 1 --2- 172.21.15.17:0/1770115158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b900b0e70 0x7f9b900afd80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9b900b87e0 tx=0x7f9b8401c7a0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.153+0000 7f9b95e52640 1 -- 172.21.15.17:0/1770115158 shutdown_connections 2024-03-16T10:11:04.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.153+0000 7f9b95e52640 1 --2- 172.21.15.17:0/1770115158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b900b0e70 0x7f9b900afd80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.153+0000 7f9b95e52640 1 --2- 172.21.15.17:0/1770115158 >> v2:172.21.15.174:3300/0 conn(0x7f9b900b2a70 0x7f9b90146f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.153+0000 7f9b95e52640 1 --2- 172.21.15.17:0/1770115158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b90143d70 0x7f9b900b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.153+0000 7f9b95e52640 1 -- 172.21.15.17:0/1770115158 >> 172.21.15.17:0/1770115158 conn(0x7f9b9006b290 msgr2=0x7f9b9006b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.153+0000 7f9b95e52640 1 -- 172.21.15.17:0/1770115158 shutdown_connections 2024-03-16T10:11:04.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.153+0000 7f9b95e52640 1 -- 172.21.15.17:0/1770115158 wait complete. 2024-03-16T10:11:04.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.153+0000 7f9b95e52640 1 Processor -- start 2024-03-16T10:11:04.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.153+0000 7f9b95e52640 1 -- start start 2024-03-16T10:11:04.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b95e52640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b900b0e70 0x7f9b9013c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b95e52640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b900b2a70 0x7f9b901dbe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b95e52640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b90143d70 0x7f9b90070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b95e52640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9b901475c0 con 0x7f9b900b2a70 2024-03-16T10:11:04.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b95e52640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b90147440 con 0x7f9b90143d70 2024-03-16T10:11:04.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b95e52640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9b9014fa40 con 0x7f9b900b0e70 2024-03-16T10:11:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b94e50640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b900b0e70 0x7f9b9013c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b95651640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b90143d70 0x7f9b90070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b94e50640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b900b0e70 0x7f9b9013c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.17:49538/0 (socket says 172.21.15.17:49538) 2024-03-16T10:11:04.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b94e50640 1 -- 172.21.15.17:0/3804678539 learned_addr learned my addr 172.21.15.17:0/3804678539 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7ff5d77fe640 1 -- 172.21.15.17:0/1413869499 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d00a5ba0 con 0x7ff5d8072d30 2024-03-16T10:11:04.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7ff5d77fe640 1 -- 172.21.15.17:0/1413869499 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00b2070 con 0x7ff5d8072d30 2024-03-16T10:11:04.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b8ffff640 1 --2- 172.21.15.17:0/3804678539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b900b2a70 0x7f9b901dbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b94e50640 1 -- 172.21.15.17:0/3804678539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b90143d70 msgr2=0x7f9b90070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b94e50640 1 --2- 172.21.15.17:0/3804678539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b90143d70 0x7f9b90070030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b94e50640 1 -- 172.21.15.17:0/3804678539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b900b2a70 msgr2=0x7f9b901dbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b94e50640 1 --2- 172.21.15.17:0/3804678539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b900b2a70 0x7f9b901dbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b94e50640 1 -- 172.21.15.17:0/3804678539 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9006f8b0 con 0x7f9b900b0e70 2024-03-16T10:11:04.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b94e50640 1 --2- 172.21.15.17:0/3804678539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b900b0e70 0x7f9b9013c8f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800b0e0 tx=0x7f9b8800b5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/1413869499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d8072d30 msgr2=0x7ff5d8151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7ff5ddb1a640 1 --2- 172.21.15.17:0/1413869499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d8072d30 0x7ff5d8151a30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80af940 tx=0x7ff5d001cad0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/1413869499 shutdown_connections 2024-03-16T10:11:04.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7ff5ddb1a640 1 --2- 172.21.15.17:0/1413869499 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d8072d30 0x7ff5d8151a30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7ff5ddb1a640 1 --2- 172.21.15.17:0/1413869499 >> v2:172.21.15.174:3300/0 conn(0x7ff5d814aff0 0x7ff5d80727f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/1413869499 >> 172.21.15.17:0/1413869499 conn(0x7ff5d806b290 msgr2=0x7ff5d806b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b8d7fa640 1 -- 172.21.15.17:0/3804678539 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8801f020 con 0x7f9b900b0e70 2024-03-16T10:11:04.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/1413869499 shutdown_connections 2024-03-16T10:11:04.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/1413869499 wait complete. 2024-03-16T10:11:04.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b95e52640 1 -- 172.21.15.17:0/3804678539 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b90071cf0 con 0x7f9b900b0e70 2024-03-16T10:11:04.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b95e52640 1 -- 172.21.15.17:0/3804678539 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9006f590 con 0x7f9b900b0e70 2024-03-16T10:11:04.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b8d7fa640 1 -- 172.21.15.17:0/3804678539 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b8801ba50 con 0x7f9b900b0e70 2024-03-16T10:11:04.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.157+0000 7f9b8d7fa640 1 -- 172.21.15.17:0/3804678539 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b8801be00 con 0x7f9b900b0e70 2024-03-16T10:11:04.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5ddb1a640 1 Processor -- start 2024-03-16T10:11:04.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5ddb1a640 1 -- start start 2024-03-16T10:11:04.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5ddb1a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5d8072d30 0x7ff5d81e7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5ddb1a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d80b5810 0x7ff5d8149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5ddb1a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5d814aff0 0x7ff5d81496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5ddb1a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d80ad8c0 con 0x7ff5d80b5810 2024-03-16T10:11:04.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5ddb1a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d80b4900 con 0x7ff5d8072d30 2024-03-16T10:11:04.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5ddb1a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d80ada40 con 0x7ff5d814aff0 2024-03-16T10:11:04.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5d7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d80b5810 0x7ff5d8149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5d7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d80b5810 0x7ff5d8149170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33784/0 (socket says 172.21.15.17:33784) 2024-03-16T10:11:04.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5d7fff640 1 -- 172.21.15.17:0/3340622559 learned_addr learned my addr 172.21.15.17:0/3340622559 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5dd319640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5d814aff0 0x7ff5d81496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5dcb18640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5d8072d30 0x7ff5d81e7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7f9b8d7fa640 1 -- 172.21.15.17:0/3804678539 <== mon.2 v2:172.21.15.174:3301/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f9b88022020 con 0x7f9b900b0e70 2024-03-16T10:11:04.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7f9b95e52640 1 -- 172.21.15.17:0/3804678539 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b9006fad0 con 0x7f9b900b0e70 2024-03-16T10:11:04.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7f9b8d7fa640 1 --2- 172.21.15.17:0/3804678539 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9b7c067660 0x7f9b7c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7f9b8d7fa640 1 -- 172.21.15.17:0/3804678539 <== mon.2 v2:172.21.15.174:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9b8808e1b0 con 0x7f9b900b0e70 2024-03-16T10:11:04.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5d7fff640 1 -- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5d814aff0 msgr2=0x7ff5d81496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5d7fff640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5d814aff0 0x7ff5d81496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5d7fff640 1 -- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5d8072d30 msgr2=0x7ff5d81e7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5d7fff640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5d8072d30 0x7ff5d81e7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7f9b8ffff640 1 --2- 172.21.15.17:0/3804678539 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9b7c067660 0x7f9b7c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5d7fff640 1 -- 172.21.15.17:0/3340622559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d8143df0 con 0x7ff5d80b5810 2024-03-16T10:11:04.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5dd319640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5d814aff0 0x7ff5d81496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:04.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7ff5d7fff640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d80b5810 0x7ff5d8149170 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00e930 tx=0x7ff5cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.161+0000 7f9b8ffff640 1 --2- 172.21.15.17:0/3804678539 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9b7c067660 0x7f9b7c069b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b9013d070 tx=0x7f9b8000a2a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.165+0000 7ff5d57fa640 1 -- 172.21.15.17:0/3340622559 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc017070 con 0x7ff5d80b5810 2024-03-16T10:11:04.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.165+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/3340622559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d8148690 con 0x7ff5d80b5810 2024-03-16T10:11:04.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.165+0000 7ff5d57fa640 1 -- 172.21.15.17:0/3340622559 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5cc00ca00 con 0x7ff5d80b5810 2024-03-16T10:11:04.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.165+0000 7ff5d57fa640 1 -- 172.21.15.17:0/3340622559 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00cdf0 con 0x7ff5d80b5810 2024-03-16T10:11:04.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.165+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/3340622559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d813ab20 con 0x7ff5d80b5810 2024-03-16T10:11:04.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.165+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/3340622559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d8068360 con 0x7ff5d80b5810 2024-03-16T10:11:04.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.165+0000 7ff5d57fa640 1 -- 172.21.15.17:0/3340622559 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7ff5cc01a070 con 0x7ff5d80b5810 2024-03-16T10:11:04.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.165+0000 7ff5d57fa640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff5c4067660 0x7ff5c4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.165+0000 7ff5d57fa640 1 -- 172.21.15.17:0/3340622559 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff5cc085860 con 0x7ff5d80b5810 2024-03-16T10:11:04.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.169+0000 7ff5dcb18640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff5c4067660 0x7ff5c4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.169+0000 7ff5dcb18640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff5c4067660 0x7ff5c4069b20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8005890 tx=0x7ff5c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.177+0000 7ff5d57fa640 1 -- 172.21.15.17:0/3340622559 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff5cc055f40 con 0x7ff5d80b5810 2024-03-16T10:11:04.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.177+0000 7f9b8d7fa640 1 -- 172.21.15.17:0/3804678539 <== mon.2 v2:172.21.15.174:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9b9006fad0 con 0x7f9b900b0e70 2024-03-16T10:11:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.169+0000 7fc4ad0a0640 1 Processor -- start 2024-03-16T10:11:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.169+0000 7fc4ad0a0640 1 -- start start 2024-03-16T10:11:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.169+0000 7fc4ad0a0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4a80b28c0 0x7fc4a80b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.169+0000 7fc4ad0a0640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc4a8157630 0x7fc4a8157a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.169+0000 7fc4ad0a0640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4a80b2ca0 con 0x7fc4a8157fb0 2024-03-16T10:11:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.169+0000 7fc4ad0a0640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fc4a80b46f0 con 0x7fc4a8157630 2024-03-16T10:11:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.169+0000 7fc4ad0a0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a8158420 con 0x7fc4a80b28c0 2024-03-16T10:11:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.169+0000 7fc4a6d9d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4a80b28c0 0x7fc4a80b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.169+0000 7fc4a6d9d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4a80b28c0 0x7fc4a80b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.17:49552/0 (socket says 172.21.15.17:49552) 2024-03-16T10:11:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.169+0000 7fc4a6d9d640 1 -- 172.21.15.17:0/1901476163 learned_addr learned my addr 172.21.15.17:0/1901476163 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.169+0000 7fc4a659c640 1 --2- 172.21.15.17:0/1901476163 >> v2:172.21.15.174:3300/0 conn(0x7fc4a8157630 0x7fc4a8157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.181+0000 7fc4a659c640 1 -- 172.21.15.17:0/1901476163 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4a80b28c0 msgr2=0x7fc4a80b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:04.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.181+0000 7fc4a659c640 1 --2- 172.21.15.17:0/1901476163 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4a80b28c0 0x7fc4a80b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.181+0000 7fc4a659c640 1 -- 172.21.15.17:0/1901476163 >> v1:172.21.15.174:6789/0 conn(0x7fc4a8157fb0 legacy=0x7fc4a81565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.181+0000 7fc4a659c640 1 -- 172.21.15.17:0/1901476163 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a80b1e60 con 0x7fc4a8157630 2024-03-16T10:11:04.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.185+0000 7fc4a659c640 1 --2- 172.21.15.17:0/1901476163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4a8157630 0x7fc4a8157a70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc49800bb50 tx=0x7fc49801e6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd696c2bb2d69833 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.185+0000 7fa92759e640 1 Processor -- start 2024-03-16T10:11:04.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.185+0000 7fa92759e640 1 -- start start 2024-03-16T10:11:04.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.185+0000 7fc4a5d9b640 1 -- 172.21.15.17:0/1901476163 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980ab470 con 0x7fc4a8157630 2024-03-16T10:11:04.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.185+0000 7fc4a5d9b640 1 -- 172.21.15.17:0/1901476163 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4980b0070 con 0x7fc4a8157630 2024-03-16T10:11:04.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.185+0000 7fc4a5d9b640 1 -- 172.21.15.17:0/1901476163 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980ab830 con 0x7fc4a8157630 2024-03-16T10:11:04.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/1901476163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4a8157630 msgr2=0x7fc4a8157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fc4ad0a0640 1 --2- 172.21.15.17:0/1901476163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4a8157630 0x7fc4a8157a70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc49800bb50 tx=0x7fc49801e6a0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fa92759e640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa9280b0e20 0x7fa9280b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fa92759e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa92806dc60 0x7fa9280b5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fa92759e640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9280b4100 con 0x7fa928146fb0 2024-03-16T10:11:04.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fa92759e640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fa9280b3280 con 0x7fa9280b0e20 2024-03-16T10:11:04.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fa92759e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa9280b3560 con 0x7fa92806dc60 2024-03-16T10:11:04.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fa92659c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa9280b0e20 0x7fa9280b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fa926d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa928146fb0 0x7fa9280b39f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56086/0 (socket says 172.21.15.17:56086) 2024-03-16T10:11:04.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fa926d9d640 1 -- 172.21.15.17:0/1240191142 learned_addr learned my addr 172.21.15.17:0/1240191142 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fa925d9b640 1 --2- 172.21.15.17:0/1240191142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa92806dc60 0x7fa9280b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fa925d9b640 1 -- 172.21.15.17:0/1240191142 >> v2:172.21.15.174:3300/0 conn(0x7fa9280b0e20 msgr2=0x7fa9280b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fa925d9b640 1 --2- 172.21.15.17:0/1240191142 >> v2:172.21.15.174:3300/0 conn(0x7fa9280b0e20 0x7fa9280b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fa925d9b640 1 -- 172.21.15.17:0/1240191142 >> v1:172.21.15.17:6789/0 conn(0x7fa928146fb0 legacy=0x7fa9280b39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fa925d9b640 1 -- 172.21.15.17:0/1240191142 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9280b12b0 con 0x7fa92806dc60 2024-03-16T10:11:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fa92559a640 1 -- 172.21.15.17:0/1240191142 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2699338532 0 0) 0x7fa9280b4100 con 0x7fa928146fb0 2024-03-16T10:11:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/1901476163 shutdown_connections 2024-03-16T10:11:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fc4ad0a0640 1 --2- 172.21.15.17:0/1901476163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4a8157630 0x7fc4a8157a70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fc4ad0a0640 1 --2- 172.21.15.17:0/1901476163 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4a80b28c0 0x7fc4a80b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/1901476163 >> 172.21.15.17:0/1901476163 conn(0x7fc4a806b460 msgr2=0x7fc4a806b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/1901476163 shutdown_connections 2024-03-16T10:11:04.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/1901476163 wait complete. 2024-03-16T10:11:04.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.189+0000 7fc4ad0a0640 1 Processor -- start 2024-03-16T10:11:04.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fa925d9b640 1 --2- 172.21.15.17:0/1240191142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa92806dc60 0x7fa9280b5980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0097f0 tx=0x7fa91c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=841d51c0b1517634 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4ad0a0640 1 -- start start 2024-03-16T10:11:04.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4ad0a0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4a80b28c0 0x7fc4a81f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4ad0a0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4a8157630 0x7fc4a81552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4ad0a0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4a8157fb0 0x7fc4a81557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4ad0a0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a80b5c70 con 0x7fc4a80b28c0 2024-03-16T10:11:04.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4ad0a0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a80afd80 con 0x7fc4a8157630 2024-03-16T10:11:04.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4ad0a0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a814c370 con 0x7fc4a8157fb0 2024-03-16T10:11:04.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4a6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4a80b28c0 0x7fc4a81f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4a6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4a80b28c0 0x7fc4a81f8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33792/0 (socket says 172.21.15.17:33792) 2024-03-16T10:11:04.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4a6d9d640 1 -- 172.21.15.17:0/2601281320 learned_addr learned my addr 172.21.15.17:0/2601281320 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fa92559a640 1 -- 172.21.15.17:0/1240191142 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0b0070 con 0x7fa92806dc60 2024-03-16T10:11:04.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fa92559a640 1 -- 172.21.15.17:0/1240191142 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa91c0ab600 con 0x7fa92806dc60 2024-03-16T10:11:04.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4a659c640 1 --2- 172.21.15.17:0/2601281320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4a8157630 0x7fc4a81552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4a759e640 1 --2- 172.21.15.17:0/2601281320 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4a8157fb0 0x7fc4a81557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4a6d9d640 1 -- 172.21.15.17:0/2601281320 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4a8157fb0 msgr2=0x7fc4a81557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4a6d9d640 1 --2- 172.21.15.17:0/2601281320 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4a8157fb0 0x7fc4a81557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4a6d9d640 1 -- 172.21.15.17:0/2601281320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4a8157630 msgr2=0x7fc4a81552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4a6d9d640 1 --2- 172.21.15.17:0/2601281320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4a8157630 0x7fc4a81552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4a6d9d640 1 -- 172.21.15.17:0/2601281320 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a8154a10 con 0x7fc4a80b28c0 2024-03-16T10:11:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4a6d9d640 1 --2- 172.21.15.17:0/2601281320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4a80b28c0 0x7fc4a81f8690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0008fb0 tx=0x7fc4a000e5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fa92559a640 1 -- 172.21.15.17:0/1240191142 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0ab9d0 con 0x7fa92806dc60 2024-03-16T10:11:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc48f7fe640 1 -- 172.21.15.17:0/2601281320 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0016640 con 0x7fc4a80b28c0 2024-03-16T10:11:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fa92759e640 1 -- 172.21.15.17:0/1240191142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa92806dc60 msgr2=0x7fa9280b5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fa92759e640 1 --2- 172.21.15.17:0/1240191142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa92806dc60 0x7fa9280b5980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa91c0097f0 tx=0x7fa91c01c4a0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.197+0000 7fa92759e640 1 -- 172.21.15.17:0/1240191142 shutdown_connections 2024-03-16T10:11:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.197+0000 7fa92759e640 1 --2- 172.21.15.17:0/1240191142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa92806dc60 0x7fa9280b5980 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.197+0000 7fa92759e640 1 --2- 172.21.15.17:0/1240191142 >> v2:172.21.15.174:3300/0 conn(0x7fa9280b0e20 0x7fa9280b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.197+0000 7fa92759e640 1 -- 172.21.15.17:0/1240191142 >> 172.21.15.17:0/1240191142 conn(0x7fa92806b460 msgr2=0x7fa92806b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/2601281320 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a8150010 con 0x7fc4a80b28c0 2024-03-16T10:11:04.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/2601281320 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a8153c20 con 0x7fc4a80b28c0 2024-03-16T10:11:04.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.197+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/2601281320 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a8068360 con 0x7fc4a80b28c0 2024-03-16T10:11:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa92759e640 1 -- 172.21.15.17:0/1240191142 shutdown_connections 2024-03-16T10:11:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa92759e640 1 -- 172.21.15.17:0/1240191142 wait complete. 2024-03-16T10:11:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa92759e640 1 Processor -- start 2024-03-16T10:11:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa92759e640 1 -- start start 2024-03-16T10:11:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa92759e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa92806dc60 0x7fa92806f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa92759e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa928146fb0 0x7fa92806fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa92759e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa928072470 0x7fa9281ec6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa92759e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa92814a620 con 0x7fa928072470 2024-03-16T10:11:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa92759e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa92814a4a0 con 0x7fa928146fb0 2024-03-16T10:11:04.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa92759e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa92814a7a0 con 0x7fa92806dc60 2024-03-16T10:11:04.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa926d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa928072470 0x7fa9281ec6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa926d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa928072470 0x7fa9281ec6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33808/0 (socket says 172.21.15.17:33808) 2024-03-16T10:11:04.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa926d9d640 1 -- 172.21.15.17:0/1837948994 learned_addr learned my addr 172.21.15.17:0/1837948994 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa925d9b640 1 --2- 172.21.15.17:0/1837948994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa928146fb0 0x7fa92806fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.205+0000 7fa92659c640 1 --2- 172.21.15.17:0/1837948994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa92806dc60 0x7fa92806f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.193+0000 7fc48f7fe640 1 -- 172.21.15.17:0/2601281320 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a00167e0 con 0x7fc4a80b28c0 2024-03-16T10:11:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.209+0000 7fa926d9d640 1 -- 172.21.15.17:0/1837948994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa92806dc60 msgr2=0x7fa92806f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.209+0000 7fa926d9d640 1 --2- 172.21.15.17:0/1837948994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa92806dc60 0x7fa92806f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.209+0000 7fa926d9d640 1 -- 172.21.15.17:0/1837948994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa928146fb0 msgr2=0x7fa92806fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.209+0000 7fa926d9d640 1 --2- 172.21.15.17:0/1837948994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa928146fb0 0x7fa92806fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.209+0000 7fa926d9d640 1 -- 172.21.15.17:0/1837948994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa928071a00 con 0x7fa928072470 2024-03-16T10:11:04.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.209+0000 7fa926d9d640 1 --2- 172.21.15.17:0/1837948994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa928072470 0x7fa9281ec6e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa92000c730 tx=0x7fa92000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.209+0000 7fc48f7fe640 1 -- 172.21.15.17:0/2601281320 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0016b90 con 0x7fc4a80b28c0 2024-03-16T10:11:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.209+0000 7fc48f7fe640 1 -- 172.21.15.17:0/2601281320 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fc4a0014050 con 0x7fc4a80b28c0 2024-03-16T10:11:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.209+0000 7fa916ffd640 1 -- 172.21.15.17:0/1837948994 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa920018070 con 0x7fa928072470 2024-03-16T10:11:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.209+0000 7fa916ffd640 1 -- 172.21.15.17:0/1837948994 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa92000ce40 con 0x7fa928072470 2024-03-16T10:11:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.209+0000 7fa916ffd640 1 -- 172.21.15.17:0/1837948994 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa92000a860 con 0x7fa928072470 2024-03-16T10:11:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.209+0000 7fa92759e640 1 -- 172.21.15.17:0/1837948994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa928071440 con 0x7fa928072470 2024-03-16T10:11:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.209+0000 7fa92759e640 1 -- 172.21.15.17:0/1837948994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa928071160 con 0x7fa928072470 2024-03-16T10:11:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.209+0000 7fa92759e640 1 -- 172.21.15.17:0/1837948994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa928068360 con 0x7fa928072470 2024-03-16T10:11:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.213+0000 7fa916ffd640 1 -- 172.21.15.17:0/1837948994 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa92001b070 con 0x7fa928072470 2024-03-16T10:11:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.213+0000 7fa916ffd640 1 --2- 172.21.15.17:0/1837948994 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa9100677f0 0x7fa910069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.213+0000 7fa916ffd640 1 -- 172.21.15.17:0/1837948994 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa920085870 con 0x7fa928072470 2024-03-16T10:11:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.213+0000 7fa92659c640 1 --2- 172.21.15.17:0/1837948994 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa9100677f0 0x7fa910069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.213+0000 7fa92659c640 1 --2- 172.21.15.17:0/1837948994 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa9100677f0 0x7fa910069cb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa928072de0 tx=0x7fa918009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.221+0000 7fc48f7fe640 1 --2- 172.21.15.17:0/2601281320 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc494067760 0x7fc494069c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.221+0000 7fc48f7fe640 1 -- 172.21.15.17:0/2601281320 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc4a00864e0 con 0x7fc4a80b28c0 2024-03-16T10:11:04.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.221+0000 7fc4a659c640 1 --2- 172.21.15.17:0/2601281320 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc494067760 0x7fc494069c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.221+0000 7fc48f7fe640 1 -- 172.21.15.17:0/2601281320 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc4a8154a10 con 0x7fc4a80b28c0 2024-03-16T10:11:04.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.221+0000 7fa916ffd640 1 -- 172.21.15.17:0/1837948994 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa920055530 con 0x7fa928072470 2024-03-16T10:11:04.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.221+0000 7fc4a659c640 1 --2- 172.21.15.17:0/2601281320 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc494067760 0x7fc494069c20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc4980af000 tx=0x7fc4980a9000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.241+0000 7ff75aa53640 1 Processor -- start 2024-03-16T10:11:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.241+0000 7ff75aa53640 1 -- start start 2024-03-16T10:11:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.241+0000 7ff75aa53640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff7540b46c0 0x7ff75414f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.241+0000 7ff75aa53640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff75406d3d0 0x7ff75406d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.241+0000 7ff75aa53640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff754073230 con 0x7ff7540aebb0 2024-03-16T10:11:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.241+0000 7ff75aa53640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ff7540ade30 con 0x7ff75406d3d0 2024-03-16T10:11:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.241+0000 7ff75aa53640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ff7540ae2c0 con 0x7ff7540b46c0 2024-03-16T10:11:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.241+0000 7ff75a252640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff75406d3d0 0x7ff75406d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.241+0000 7ff75a252640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ff75406d3d0 0x7ff75406d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33820/0 (socket says 172.21.15.17:33820) 2024-03-16T10:11:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.241+0000 7ff75a252640 1 -- 172.21.15.17:0/2916435420 learned_addr learned my addr 172.21.15.17:0/2916435420 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.241+0000 7ff75a252640 1 -- 172.21.15.17:0/2916435420 >> v2:172.21.15.174:3300/0 conn(0x7ff7540b46c0 msgr2=0x7ff75414f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:04.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.241+0000 7ff759a51640 1 --2- 172.21.15.17:0/2916435420 >> v2:172.21.15.174:3300/0 conn(0x7ff7540b46c0 0x7ff75414f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.245+0000 7ff75a252640 1 --2- 172.21.15.17:0/2916435420 >> v2:172.21.15.174:3300/0 conn(0x7ff7540b46c0 0x7ff75414f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.245+0000 7ff75a252640 1 -- 172.21.15.17:0/2916435420 >> v1:172.21.15.174:6789/0 conn(0x7ff7540aebb0 legacy=0x7ff75414f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.245+0000 7ff75a252640 1 -- 172.21.15.17:0/2916435420 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7540b4aa0 con 0x7ff75406d3d0 2024-03-16T10:11:04.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.245+0000 7ff759a51640 1 --2- 172.21.15.17:0/2916435420 >> v2:172.21.15.174:3300/0 conn(0x7ff7540b46c0 0x7ff75414f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:04.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.245+0000 7ff75a252640 1 --2- 172.21.15.17:0/2916435420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff75406d3d0 0x7ff75406d7e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff7540b65f0 tx=0x7ff75001bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac2bf44a42d5f1d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.245+0000 7ff758a4f640 1 -- 172.21.15.17:0/2916435420 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7500b1070 con 0x7ff75406d3d0 2024-03-16T10:11:04.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.245+0000 7ff758a4f640 1 -- 172.21.15.17:0/2916435420 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7500a4740 con 0x7ff75406d3d0 2024-03-16T10:11:04.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.245+0000 7ff758a4f640 1 -- 172.21.15.17:0/2916435420 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7500a4b10 con 0x7ff75406d3d0 2024-03-16T10:11:04.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.249+0000 7ff75aa53640 1 -- 172.21.15.17:0/2916435420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff75406d3d0 msgr2=0x7ff75406d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.249+0000 7ff75aa53640 1 --2- 172.21.15.17:0/2916435420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff75406d3d0 0x7ff75406d7e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff7540b65f0 tx=0x7ff75001bfa0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75aa53640 1 -- 172.21.15.17:0/2916435420 shutdown_connections 2024-03-16T10:11:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75aa53640 1 --2- 172.21.15.17:0/2916435420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff75406d3d0 0x7ff75406d7e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75aa53640 1 --2- 172.21.15.17:0/2916435420 >> v2:172.21.15.174:3300/0 conn(0x7ff7540b46c0 0x7ff75414f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75aa53640 1 -- 172.21.15.17:0/2916435420 >> 172.21.15.17:0/2916435420 conn(0x7ff7540b5550 msgr2=0x7ff7540b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75aa53640 1 -- 172.21.15.17:0/2916435420 shutdown_connections 2024-03-16T10:11:04.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75aa53640 1 -- 172.21.15.17:0/2916435420 wait complete. 2024-03-16T10:11:04.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75aa53640 1 Processor -- start 2024-03-16T10:11:04.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75aa53640 1 -- start start 2024-03-16T10:11:04.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75aa53640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff75406d3d0 0x7ff7540711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75aa53640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7540aebb0 0x7ff754135100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75aa53640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7540b46c0 0x7ff754135640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75aa53640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff754157360 con 0x7ff7540aebb0 2024-03-16T10:11:04.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75aa53640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7541502d0 con 0x7ff7540b46c0 2024-03-16T10:11:04.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75aa53640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff7541574e0 con 0x7ff75406d3d0 2024-03-16T10:11:04.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff759250640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7540aebb0 0x7ff754135100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff759250640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7540aebb0 0x7ff754135100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33838/0 (socket says 172.21.15.17:33838) 2024-03-16T10:11:04.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff759250640 1 -- 172.21.15.17:0/1934603158 learned_addr learned my addr 172.21.15.17:0/1934603158 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff759a51640 1 --2- 172.21.15.17:0/1934603158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff75406d3d0 0x7ff7540711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff75a252640 1 --2- 172.21.15.17:0/1934603158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7540b46c0 0x7ff754135640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff759250640 1 -- 172.21.15.17:0/1934603158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff75406d3d0 msgr2=0x7ff7540711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff759250640 1 --2- 172.21.15.17:0/1934603158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff75406d3d0 0x7ff7540711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff759250640 1 -- 172.21.15.17:0/1934603158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7540b46c0 msgr2=0x7ff754135640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff759250640 1 --2- 172.21.15.17:0/1934603158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7540b46c0 0x7ff754135640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.257+0000 7ff759250640 1 -- 172.21.15.17:0/1934603158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7540b70e0 con 0x7ff7540aebb0 2024-03-16T10:11:04.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.261+0000 7ff759250640 1 --2- 172.21.15.17:0/1934603158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7540aebb0 0x7ff754135100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00caf0 tx=0x7ff74c002180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7ff74a7fc640 1 -- 172.21.15.17:0/1934603158 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c017070 con 0x7ff7540aebb0 2024-03-16T10:11:04.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7ff75aa53640 1 -- 172.21.15.17:0/1934603158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75409b660 con 0x7ff7540aebb0 2024-03-16T10:11:04.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.253+0000 7f2df2e8d640 1 Processor -- start 2024-03-16T10:11:04.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.253+0000 7f2df2e8d640 1 -- start start 2024-03-16T10:11:04.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.253+0000 7f2df2e8d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2de4036c10 0x7f2de40eab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.253+0000 7f2df2e8d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2de4038560 0x7f2de40e3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.253+0000 7f2df2e8d640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2de4037e40 con 0x7f2de4036130 2024-03-16T10:11:04.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.253+0000 7f2df2e8d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f2de40ea6a0 con 0x7f2de4036c10 2024-03-16T10:11:04.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.253+0000 7f2df2e8d640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f2de40ea330 con 0x7f2de4038560 2024-03-16T10:11:04.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.253+0000 7f2df1e8b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2de4036c10 0x7f2de40eab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.253+0000 7f2df1e8b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2de4036c10 0x7f2de40eab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33834/0 (socket says 172.21.15.17:33834) 2024-03-16T10:11:04.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.253+0000 7f2df1e8b640 1 -- 172.21.15.17:0/1824256784 learned_addr learned my addr 172.21.15.17:0/1824256784 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.253+0000 7f2df168a640 1 --2- 172.21.15.17:0/1824256784 >> v2:172.21.15.174:3300/0 conn(0x7f2de4038560 0x7f2de40e3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.253+0000 7f2df0e89640 1 -- 172.21.15.17:0/1824256784 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2792239640 0 0) 0x7f2de4037e40 con 0x7f2de4036130 2024-03-16T10:11:04.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7f2df0e89640 1 -- 172.21.15.17:0/1824256784 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2dd8003530 con 0x7f2de4036130 2024-03-16T10:11:04.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7ff75aa53640 1 -- 172.21.15.17:0/1934603158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff754138100 con 0x7ff7540aebb0 2024-03-16T10:11:04.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7ff74a7fc640 1 -- 172.21.15.17:0/1934603158 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff74c003db0 con 0x7ff7540aebb0 2024-03-16T10:11:04.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7ff74a7fc640 1 -- 172.21.15.17:0/1934603158 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c00a9c0 con 0x7ff7540aebb0 2024-03-16T10:11:04.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7ff74a7fc640 1 -- 172.21.15.17:0/1934603158 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7ff74c01a070 con 0x7ff7540aebb0 2024-03-16T10:11:04.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7f2df1e8b640 1 -- 172.21.15.17:0/1824256784 >> v2:172.21.15.174:3300/0 conn(0x7f2de4038560 msgr2=0x7f2de40e3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:04.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7f2df1e8b640 1 --2- 172.21.15.17:0/1824256784 >> v2:172.21.15.174:3300/0 conn(0x7f2de4038560 0x7f2de40e3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7f2df1e8b640 1 -- 172.21.15.17:0/1824256784 >> v1:172.21.15.174:6789/0 conn(0x7f2de4036130 legacy=0x7f2de40ed710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7f2df1e8b640 1 -- 172.21.15.17:0/1824256784 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de40eb070 con 0x7f2de4036c10 2024-03-16T10:11:04.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7f2df1e8b640 1 --2- 172.21.15.17:0/1824256784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de4036c10 0x7f2de40eab30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2de4035100 tx=0x7f2de801e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1471b9941b09bf8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7f2df0e89640 1 -- 172.21.15.17:0/1824256784 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de80b5070 con 0x7f2de4036c10 2024-03-16T10:11:04.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7f2df0e89640 1 -- 172.21.15.17:0/1824256784 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2de80a8040 con 0x7f2de4036c10 2024-03-16T10:11:04.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.265+0000 7f2df0e89640 1 -- 172.21.15.17:0/1824256784 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8008e00 con 0x7f2de4036c10 2024-03-16T10:11:04.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7ff75aa53640 1 -- 172.21.15.17:0/1934603158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff73c0052a0 con 0x7ff7540aebb0 2024-03-16T10:11:04.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 -- 172.21.15.17:0/1824256784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de4036c10 msgr2=0x7f2de40eab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 --2- 172.21.15.17:0/1824256784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de4036c10 0x7f2de40eab30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2de4035100 tx=0x7f2de801e5d0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 -- 172.21.15.17:0/1824256784 shutdown_connections 2024-03-16T10:11:04.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 --2- 172.21.15.17:0/1824256784 >> v2:172.21.15.174:3300/0 conn(0x7f2de4038560 0x7f2de40e3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 --2- 172.21.15.17:0/1824256784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de4036c10 0x7f2de40eab30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 -- 172.21.15.17:0/1824256784 >> 172.21.15.17:0/1824256784 conn(0x7f2de403a830 msgr2=0x7f2de403ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 -- 172.21.15.17:0/1824256784 shutdown_connections 2024-03-16T10:11:04.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 -- 172.21.15.17:0/1824256784 wait complete. 2024-03-16T10:11:04.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 Processor -- start 2024-03-16T10:11:04.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 -- start start 2024-03-16T10:11:04.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2de4036130 0x7f2de40e92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de4036c10 0x7f2de417f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2de4038560 0x7f2de417f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2de40f7e50 con 0x7f2de4036c10 2024-03-16T10:11:04.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2de40f7cd0 con 0x7f2de4036130 2024-03-16T10:11:04.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.269+0000 7f2df2e8d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2de40f7fd0 con 0x7f2de4038560 2024-03-16T10:11:04.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.273+0000 7f2df168a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de4036c10 0x7f2de417f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.273+0000 7f2df168a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de4036c10 0x7f2de417f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33844/0 (socket says 172.21.15.17:33844) 2024-03-16T10:11:04.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.273+0000 7f2df168a640 1 -- 172.21.15.17:0/3140309637 learned_addr learned my addr 172.21.15.17:0/3140309637 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.273+0000 7f2df1e8b640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2de4036130 0x7f2de40e92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.273+0000 7f2df40ef640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2de4038560 0x7f2de417f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.273+0000 7f2df168a640 1 -- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2de4038560 msgr2=0x7f2de417f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.273+0000 7f2df168a640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2de4038560 0x7f2de417f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.273+0000 7f2df168a640 1 -- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2de4036130 msgr2=0x7f2de40e92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.273+0000 7f2df168a640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2de4036130 0x7f2de40e92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.273+0000 7f2df168a640 1 -- 172.21.15.17:0/3140309637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de417d5a0 con 0x7f2de4036c10 2024-03-16T10:11:04.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.273+0000 7f2df1e8b640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2de4036130 0x7f2de40e92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:11:04.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.273+0000 7f2df168a640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de4036c10 0x7f2de417f230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00eb80 tx=0x7f2ddc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.277+0000 7ff74a7fc640 1 --2- 172.21.15.17:0/1934603158 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff738067760 0x7ff738069c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.277+0000 7ff74a7fc640 1 -- 172.21.15.17:0/1934603158 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff74c085210 con 0x7ff7540aebb0 2024-03-16T10:11:04.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.277+0000 7ff74a7fc640 1 -- 172.21.15.17:0/1934603158 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff74c08b1d0 con 0x7ff7540aebb0 2024-03-16T10:11:04.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.277+0000 7f2de27fc640 1 -- 172.21.15.17:0/3140309637 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc01c070 con 0x7f2de4036c10 2024-03-16T10:11:04.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.277+0000 7ff759a51640 1 --2- 172.21.15.17:0/1934603158 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff738067760 0x7ff738069c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.277+0000 7f2df2e8d640 1 -- 172.21.15.17:0/3140309637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de417eda0 con 0x7f2de4036c10 2024-03-16T10:11:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.277+0000 7f2df2e8d640 1 -- 172.21.15.17:0/3140309637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de417eac0 con 0x7f2de4036c10 2024-03-16T10:11:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.277+0000 7f2de27fc640 1 -- 172.21.15.17:0/3140309637 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ddc00cd10 con 0x7f2de4036c10 2024-03-16T10:11:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.277+0000 7f2de27fc640 1 -- 172.21.15.17:0/3140309637 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc026060 con 0x7f2de4036c10 2024-03-16T10:11:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.281+0000 7f2de27fc640 1 -- 172.21.15.17:0/3140309637 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f2ddc01f070 con 0x7f2de4036c10 2024-03-16T10:11:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.281+0000 7f2de27fc640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2dd0067660 0x7f2dd0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.281+0000 7f2de27fc640 1 -- 172.21.15.17:0/3140309637 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2ddc085c50 con 0x7f2de4036c10 2024-03-16T10:11:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.281+0000 7f2df1e8b640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2dd0067660 0x7f2dd0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.281+0000 7f2df1e8b640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2dd0067660 0x7f2dd0069b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2de80b2000 tx=0x7f2de80a9000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.281+0000 7ff759a51640 1 --2- 172.21.15.17:0/1934603158 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff738067760 0x7ff738069c20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff74400ab60 tx=0x7ff744008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.293+0000 7f2df2e8d640 1 -- 172.21.15.17:0/3140309637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd40052a0 con 0x7f2de4036c10 2024-03-16T10:11:04.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a93053640 1 Processor -- start 2024-03-16T10:11:04.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a93053640 1 -- start start 2024-03-16T10:11:04.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f2de27fc640 1 -- 172.21.15.17:0/3140309637 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2ddc056290 con 0x7f2de4036c10 2024-03-16T10:11:04.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a93053640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a8400c360 0x7f9a840d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a93053640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a840d0f80 0x7f9a840db320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a93053640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a8400c740 con 0x7f9a8401f1a0 2024-03-16T10:11:04.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a93053640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9a84007bd0 con 0x7f9a8400c360 2024-03-16T10:11:04.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a93053640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9a84008470 con 0x7f9a840d0f80 2024-03-16T10:11:04.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a92051640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a8400c360 0x7f9a840d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a92852640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a840d0f80 0x7f9a840db320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a92051640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a8400c360 0x7f9a840d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33860/0 (socket says 172.21.15.17:33860) 2024-03-16T10:11:04.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a92051640 1 -- 172.21.15.17:0/1066636931 learned_addr learned my addr 172.21.15.17:0/1066636931 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a92852640 1 -- 172.21.15.17:0/1066636931 >> v2:172.21.15.17:3300/0 conn(0x7f9a8400c360 msgr2=0x7f9a840d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:04.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a92852640 1 --2- 172.21.15.17:0/1066636931 >> v2:172.21.15.17:3300/0 conn(0x7f9a8400c360 0x7f9a840d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a92852640 1 -- 172.21.15.17:0/1066636931 >> v1:172.21.15.17:6789/0 conn(0x7f9a8401f1a0 legacy=0x7f9a8401e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:04.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.301+0000 7f9a92852640 1 -- 172.21.15.17:0/1066636931 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a84007e20 con 0x7f9a840d0f80 2024-03-16T10:11:04.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.305+0000 7f9a92051640 1 --2- 172.21.15.17:0/1066636931 >> v2:172.21.15.17:3300/0 conn(0x7f9a8400c360 0x7f9a840d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:11:04.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.305+0000 7f9a92852640 1 --2- 172.21.15.17:0/1066636931 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a840d0f80 0x7f9a840db320 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c04fa70 tx=0x7f9a8c097f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19126ddcad8c3441 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.305+0000 7f9a9104f640 1 -- 172.21.15.17:0/1066636931 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c120d40 con 0x7f9a840d0f80 2024-03-16T10:11:04.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.305+0000 7f9a9104f640 1 -- 172.21.15.17:0/1066636931 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a8c128440 con 0x7f9a840d0f80 2024-03-16T10:11:04.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.305+0000 7f9a9104f640 1 -- 172.21.15.17:0/1066636931 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c1288f0 con 0x7f9a840d0f80 2024-03-16T10:11:04.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 -- 172.21.15.17:0/1066636931 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a840d0f80 msgr2=0x7f9a840db320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 --2- 172.21.15.17:0/1066636931 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a840d0f80 0x7f9a840db320 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c04fa70 tx=0x7f9a8c097f90 comp rx=0 tx=0).stop 2024-03-16T10:11:04.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 -- 172.21.15.17:0/1066636931 shutdown_connections 2024-03-16T10:11:04.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 --2- 172.21.15.17:0/1066636931 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a840d0f80 0x7f9a840db320 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 --2- 172.21.15.17:0/1066636931 >> v2:172.21.15.17:3300/0 conn(0x7f9a8400c360 0x7f9a840d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 -- 172.21.15.17:0/1066636931 >> 172.21.15.17:0/1066636931 conn(0x7f9a8400b5e0 msgr2=0x7f9a8400b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 -- 172.21.15.17:0/1066636931 shutdown_connections 2024-03-16T10:11:04.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 -- 172.21.15.17:0/1066636931 wait complete. 2024-03-16T10:11:04.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 Processor -- start 2024-03-16T10:11:04.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 -- start start 2024-03-16T10:11:04.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a8400c360 0x7f9a84027890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a8401f1a0 0x7f9a840289e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a840d0f80 0x7f9a84026370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9a840dc440 con 0x7f9a840d0f80 2024-03-16T10:11:04.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9a840dc2c0 con 0x7f9a8400c360 2024-03-16T10:11:04.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a93053640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9a840dc5c0 con 0x7f9a8401f1a0 2024-03-16T10:11:04.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a92852640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a840d0f80 0x7f9a84026370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a92852640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a840d0f80 0x7f9a84026370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33864/0 (socket says 172.21.15.17:33864) 2024-03-16T10:11:04.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a92852640 1 -- 172.21.15.17:0/815495100 learned_addr learned my addr 172.21.15.17:0/815495100 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.309+0000 7f9a91850640 1 --2- 172.21.15.17:0/815495100 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a8401f1a0 0x7f9a840289e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.313+0000 7f9a92852640 1 -- 172.21.15.17:0/815495100 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a8401f1a0 msgr2=0x7f9a840289e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.313+0000 7f9a92852640 1 --2- 172.21.15.17:0/815495100 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a8401f1a0 0x7f9a840289e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.313+0000 7f9a92852640 1 -- 172.21.15.17:0/815495100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a8400c360 msgr2=0x7f9a84027890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.313+0000 7f9a92852640 1 --2- 172.21.15.17:0/815495100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a8400c360 0x7f9a84027890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.313+0000 7f9a92852640 1 -- 172.21.15.17:0/815495100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a84028260 con 0x7f9a840d0f80 2024-03-16T10:11:04.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.313+0000 7f9a92852640 1 --2- 172.21.15.17:0/815495100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a840d0f80 0x7f9a84026370 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c050860 tx=0x7f9a8c094dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.321+0000 7f9a82ffd640 1 -- 172.21.15.17:0/815495100 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c138020 con 0x7f9a840d0f80 2024-03-16T10:11:04.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.321+0000 7f9a82ffd640 1 -- 172.21.15.17:0/815495100 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a8c129c40 con 0x7f9a840d0f80 2024-03-16T10:11:04.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.321+0000 7f9a93053640 1 -- 172.21.15.17:0/815495100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a84027f80 con 0x7f9a840d0f80 2024-03-16T10:11:04.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.321+0000 7f9a82ffd640 1 -- 172.21.15.17:0/815495100 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c134670 con 0x7f9a840d0f80 2024-03-16T10:11:04.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.321+0000 7f9a93053640 1 -- 172.21.15.17:0/815495100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a840268b0 con 0x7f9a840d0f80 2024-03-16T10:11:04.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.329+0000 7f9a82ffd640 1 -- 172.21.15.17:0/815495100 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f9a8c13b020 con 0x7f9a840d0f80 2024-03-16T10:11:04.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.329+0000 7f9a82ffd640 1 --2- 172.21.15.17:0/815495100 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a70067730 0x7f9a70069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.329+0000 7f9a82ffd640 1 -- 172.21.15.17:0/815495100 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9a8c19b0b0 con 0x7f9a840d0f80 2024-03-16T10:11:04.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.329+0000 7f9a93053640 1 -- 172.21.15.17:0/815495100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a840057f0 con 0x7f9a840d0f80 2024-03-16T10:11:04.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.329+0000 7f9a92051640 1 --2- 172.21.15.17:0/815495100 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a70067730 0x7f9a70069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.329+0000 7f9a92051640 1 --2- 172.21.15.17:0/815495100 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a70067730 0x7f9a70069bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9a88006f50 tx=0x7f9a88006ea0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.337+0000 7f9a82ffd640 1 -- 172.21.15.17:0/815495100 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9a8c091d70 con 0x7f9a840d0f80 2024-03-16T10:11:04.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.481+0000 7ffa79a6f640 1 -- 172.21.15.17:0/2225689859 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ffa54005590 con 0x7ffa740b0e70 2024-03-16T10:11:04.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.481+0000 7ffa717fa640 1 -- 172.21.15.17:0/2225689859 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffa640558d0 con 0x7ffa740b0e70 2024-03-16T10:11:04.488 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-16T10:11:04.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.485+0000 7ffa5affd640 1 -- 172.21.15.17:0/2225689859 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ffa60067730 msgr2=0x7ffa60069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.485+0000 7ffa5affd640 1 --2- 172.21.15.17:0/2225689859 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ffa60067730 0x7ffa60069bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0af000 tx=0x7ffa6c0a8070 comp rx=0 tx=0).stop 2024-03-16T10:11:04.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.485+0000 7ffa5affd640 1 -- 172.21.15.17:0/2225689859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa740b0e70 msgr2=0x7ffa740a95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.485+0000 7ffa5affd640 1 --2- 172.21.15.17:0/2225689859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa740b0e70 0x7ffa740a95d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffa6400cc10 tx=0x7ffa6400a430 comp rx=0 tx=0).stop 2024-03-16T10:11:04.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.485+0000 7ffa5affd640 1 -- 172.21.15.17:0/2225689859 shutdown_connections 2024-03-16T10:11:04.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.485+0000 7ffa5affd640 1 --2- 172.21.15.17:0/2225689859 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ffa60067730 0x7ffa60069bf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.485+0000 7ffa5affd640 1 --2- 172.21.15.17:0/2225689859 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffa74143d70 0x7ffa741e1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.485+0000 7ffa5affd640 1 --2- 172.21.15.17:0/2225689859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa740b2a70 0x7ffa741e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.485+0000 7ffa5affd640 1 --2- 172.21.15.17:0/2225689859 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa740b0e70 0x7ffa740a95d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.485+0000 7ffa5affd640 1 -- 172.21.15.17:0/2225689859 >> 172.21.15.17:0/2225689859 conn(0x7ffa7406b290 msgr2=0x7ffa741403d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.485+0000 7ffa5affd640 1 -- 172.21.15.17:0/2225689859 shutdown_connections 2024-03-16T10:11:04.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.485+0000 7ffa5affd640 1 -- 172.21.15.17:0/2225689859 wait complete. 2024-03-16T10:11:04.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.501+0000 7f9b95e52640 1 -- 172.21.15.17:0/3804678539 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f9b90068360 con 0x7f9b900b0e70 2024-03-16T10:11:04.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.501+0000 7f9b8d7fa640 1 -- 172.21.15.17:0/3804678539 <== mon.2 v2:172.21.15.174:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9b88029070 con 0x7f9b900b0e70 2024-03-16T10:11:04.508 INFO:teuthology.orchestra.run.smithi017.stdout:55834574853 2024-03-16T10:11:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.505+0000 7f9b76ffd640 1 -- 172.21.15.17:0/3804678539 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9b7c067660 msgr2=0x7f9b7c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.505+0000 7f9b76ffd640 1 --2- 172.21.15.17:0/3804678539 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9b7c067660 0x7f9b7c069b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b9013d070 tx=0x7f9b8000a2a0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.505+0000 7f9b76ffd640 1 -- 172.21.15.17:0/3804678539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b900b0e70 msgr2=0x7f9b9013c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.505+0000 7f9b76ffd640 1 --2- 172.21.15.17:0/3804678539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b900b0e70 0x7f9b9013c8f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9b8800b0e0 tx=0x7f9b8800b5a0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.505+0000 7f9b76ffd640 1 -- 172.21.15.17:0/3804678539 shutdown_connections 2024-03-16T10:11:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.505+0000 7f9b76ffd640 1 --2- 172.21.15.17:0/3804678539 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9b7c067660 0x7f9b7c069b20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.505+0000 7f9b76ffd640 1 --2- 172.21.15.17:0/3804678539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b90143d70 0x7f9b90070030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.505+0000 7f9b76ffd640 1 --2- 172.21.15.17:0/3804678539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b900b2a70 0x7f9b901dbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.505+0000 7f9b76ffd640 1 --2- 172.21.15.17:0/3804678539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b900b0e70 0x7f9b9013c8f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.505+0000 7f9b76ffd640 1 -- 172.21.15.17:0/3804678539 >> 172.21.15.17:0/3804678539 conn(0x7f9b9006b290 msgr2=0x7f9b901338c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.505+0000 7f9b76ffd640 1 -- 172.21.15.17:0/3804678539 shutdown_connections 2024-03-16T10:11:04.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.505+0000 7f9b76ffd640 1 -- 172.21.15.17:0/3804678539 wait complete. 2024-03-16T10:11:04.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2024-03-16T10:11:04.512 DEBUG:teuthology.parallel:result is None 2024-03-16T10:11:04.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.513+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/3340622559 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff5d80b1820 con 0x7ff5d80b5810 2024-03-16T10:11:04.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.517+0000 7ff5d57fa640 1 -- 172.21.15.17:0/3340622559 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff5cc059fe0 con 0x7ff5d80b5810 2024-03-16T10:11:04.519 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-16T10:11:04.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.517+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff5c4067660 msgr2=0x7ff5c4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.517+0000 7ff5ddb1a640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff5c4067660 0x7ff5c4069b20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8005890 tx=0x7ff5c8009040 comp rx=0 tx=0).stop 2024-03-16T10:11:04.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.517+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/3340622559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d80b5810 msgr2=0x7ff5d8149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.517+0000 7ff5ddb1a640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d80b5810 0x7ff5d8149170 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc00e930 tx=0x7ff5cc00edf0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.517+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/3340622559 shutdown_connections 2024-03-16T10:11:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.517+0000 7ff5ddb1a640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff5c4067660 0x7ff5c4069b20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.517+0000 7ff5ddb1a640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff5d814aff0 0x7ff5d81496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.517+0000 7ff5ddb1a640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5d80b5810 0x7ff5d8149170 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.517+0000 7ff5ddb1a640 1 --2- 172.21.15.17:0/3340622559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5d8072d30 0x7ff5d81e7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.517+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/3340622559 >> 172.21.15.17:0/3340622559 conn(0x7ff5d806b290 msgr2=0x7ff5d80b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.517+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/3340622559 shutdown_connections 2024-03-16T10:11:04.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.517+0000 7ff5ddb1a640 1 -- 172.21.15.17:0/3340622559 wait complete. 2024-03-16T10:11:04.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2024-03-16T10:11:04.533 DEBUG:teuthology.parallel:result is None 2024-03-16T10:11:04.545 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-03-16T10:11:04.545 DEBUG:teuthology.parallel:result is None 2024-03-16T10:11:04.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.545+0000 7fa92759e640 1 -- 172.21.15.17:0/1837948994 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa928070390 con 0x7fa928072470 2024-03-16T10:11:04.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.545+0000 7fa916ffd640 1 -- 172.21.15.17:0/1837948994 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa9200595d0 con 0x7fa928072470 2024-03-16T10:11:04.549 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-16T10:11:04.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.545+0000 7fa92759e640 1 -- 172.21.15.17:0/1837948994 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa9100677f0 msgr2=0x7fa910069cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.545+0000 7fa92759e640 1 --2- 172.21.15.17:0/1837948994 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa9100677f0 0x7fa910069cb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa928072de0 tx=0x7fa918009190 comp rx=0 tx=0).stop 2024-03-16T10:11:04.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.545+0000 7fa92759e640 1 -- 172.21.15.17:0/1837948994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa928072470 msgr2=0x7fa9281ec6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.545+0000 7fa92759e640 1 --2- 172.21.15.17:0/1837948994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa928072470 0x7fa9281ec6e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa92000c730 tx=0x7fa92000cbf0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.545+0000 7fa92759e640 1 -- 172.21.15.17:0/1837948994 shutdown_connections 2024-03-16T10:11:04.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.545+0000 7fa92759e640 1 --2- 172.21.15.17:0/1837948994 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa9100677f0 0x7fa910069cb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.545+0000 7fa92759e640 1 --2- 172.21.15.17:0/1837948994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa928072470 0x7fa9281ec6e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.545+0000 7fa92759e640 1 --2- 172.21.15.17:0/1837948994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa928146fb0 0x7fa92806fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.545+0000 7fa92759e640 1 --2- 172.21.15.17:0/1837948994 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa92806dc60 0x7fa92806f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.545+0000 7fa92759e640 1 -- 172.21.15.17:0/1837948994 >> 172.21.15.17:0/1837948994 conn(0x7fa92806b460 msgr2=0x7fa92814c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.545+0000 7fa92759e640 1 -- 172.21.15.17:0/1837948994 shutdown_connections 2024-03-16T10:11:04.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.545+0000 7fa92759e640 1 -- 172.21.15.17:0/1837948994 wait complete. 2024-03-16T10:11:04.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.557+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/2601281320 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc4a8072e90 con 0x7fc4a80b28c0 2024-03-16T10:11:04.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.557+0000 7fc48f7fe640 1 -- 172.21.15.17:0/2601281320 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc4a0056b80 con 0x7fc4a80b28c0 2024-03-16T10:11:04.561 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-16T10:11:04.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.561+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/2601281320 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc494067760 msgr2=0x7fc494069c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.561+0000 7fc4ad0a0640 1 --2- 172.21.15.17:0/2601281320 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc494067760 0x7fc494069c20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc4980af000 tx=0x7fc4980a9000 comp rx=0 tx=0).stop 2024-03-16T10:11:04.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.561+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/2601281320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4a80b28c0 msgr2=0x7fc4a81f8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.561+0000 7fc4ad0a0640 1 --2- 172.21.15.17:0/2601281320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4a80b28c0 0x7fc4a81f8690 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0008fb0 tx=0x7fc4a000e5c0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.561+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/2601281320 shutdown_connections 2024-03-16T10:11:04.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.561+0000 7fc4ad0a0640 1 --2- 172.21.15.17:0/2601281320 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc494067760 0x7fc494069c20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.561+0000 7fc4ad0a0640 1 --2- 172.21.15.17:0/2601281320 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4a8157fb0 0x7fc4a81557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.561+0000 7fc4ad0a0640 1 --2- 172.21.15.17:0/2601281320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4a8157630 0x7fc4a81552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.561+0000 7fc4ad0a0640 1 --2- 172.21.15.17:0/2601281320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4a80b28c0 0x7fc4a81f8690 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.561+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/2601281320 >> 172.21.15.17:0/2601281320 conn(0x7fc4a806b460 msgr2=0x7fc4a80b8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.561+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/2601281320 shutdown_connections 2024-03-16T10:11:04.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.561+0000 7fc4ad0a0640 1 -- 172.21.15.17:0/2601281320 wait complete. 2024-03-16T10:11:04.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-03-16T10:11:04.573 DEBUG:teuthology.parallel:result is None 2024-03-16T10:11:04.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.573+0000 7ff75aa53640 1 -- 172.21.15.17:0/1934603158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7ff73c005590 con 0x7ff7540aebb0 2024-03-16T10:11:04.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.573+0000 7ff74a7fc640 1 -- 172.21.15.17:0/1934603158 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff74c0558b0 con 0x7ff7540aebb0 2024-03-16T10:11:04.578 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-16T10:11:04.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.577+0000 7ff75aa53640 1 -- 172.21.15.17:0/1934603158 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff738067760 msgr2=0x7ff738069c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.577+0000 7ff75aa53640 1 --2- 172.21.15.17:0/1934603158 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff738067760 0x7ff738069c20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff74400ab60 tx=0x7ff744008040 comp rx=0 tx=0).stop 2024-03-16T10:11:04.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.577+0000 7ff75aa53640 1 -- 172.21.15.17:0/1934603158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7540aebb0 msgr2=0x7ff754135100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.577+0000 7ff75aa53640 1 --2- 172.21.15.17:0/1934603158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7540aebb0 0x7ff754135100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00caf0 tx=0x7ff74c002180 comp rx=0 tx=0).stop 2024-03-16T10:11:04.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.577+0000 7ff75aa53640 1 -- 172.21.15.17:0/1934603158 shutdown_connections 2024-03-16T10:11:04.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.577+0000 7ff75aa53640 1 --2- 172.21.15.17:0/1934603158 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff738067760 0x7ff738069c20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.577+0000 7ff75aa53640 1 --2- 172.21.15.17:0/1934603158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7540b46c0 0x7ff754135640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.577+0000 7ff75aa53640 1 --2- 172.21.15.17:0/1934603158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7540aebb0 0x7ff754135100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.577+0000 7ff75aa53640 1 --2- 172.21.15.17:0/1934603158 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff75406d3d0 0x7ff7540711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.577+0000 7ff75aa53640 1 -- 172.21.15.17:0/1934603158 >> 172.21.15.17:0/1934603158 conn(0x7ff7540b5550 msgr2=0x7ff7540b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.577+0000 7ff75aa53640 1 -- 172.21.15.17:0/1934603158 shutdown_connections 2024-03-16T10:11:04.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.577+0000 7ff75aa53640 1 -- 172.21.15.17:0/1934603158 wait complete. 2024-03-16T10:11:04.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-03-16T10:11:04.587 DEBUG:teuthology.parallel:result is None 2024-03-16T10:11:04.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-03-16T10:11:04.597 DEBUG:teuthology.parallel:result is None 2024-03-16T10:11:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.621+0000 7f2df2e8d640 1 -- 172.21.15.17:0/3140309637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2dd4005860 con 0x7f2de4036c10 2024-03-16T10:11:04.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.621+0000 7f2de27fc640 1 -- 172.21.15.17:0/3140309637 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2ddc05a330 con 0x7f2de4036c10 2024-03-16T10:11:04.626 INFO:teuthology.orchestra.run.smithi017.stdout:55834574854 2024-03-16T10:11:04.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.625+0000 7f2df2e8d640 1 -- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2dd0067660 msgr2=0x7f2dd0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.625+0000 7f2df2e8d640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2dd0067660 0x7f2dd0069b20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2de80b2000 tx=0x7f2de80a9000 comp rx=0 tx=0).stop 2024-03-16T10:11:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.625+0000 7f2df2e8d640 1 -- 172.21.15.17:0/3140309637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de4036c10 msgr2=0x7f2de417f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.625+0000 7f2df2e8d640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de4036c10 0x7f2de417f230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc00eb80 tx=0x7f2ddc00ec70 comp rx=0 tx=0).stop 2024-03-16T10:11:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.625+0000 7f2df2e8d640 1 -- 172.21.15.17:0/3140309637 shutdown_connections 2024-03-16T10:11:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.625+0000 7f2df2e8d640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2dd0067660 0x7f2dd0069b20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.625+0000 7f2df2e8d640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2de4038560 0x7f2de417f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.625+0000 7f2df2e8d640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2de4036c10 0x7f2de417f230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.625+0000 7f2df2e8d640 1 --2- 172.21.15.17:0/3140309637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2de4036130 0x7f2de40e92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.625+0000 7f2df2e8d640 1 -- 172.21.15.17:0/3140309637 >> 172.21.15.17:0/3140309637 conn(0x7f2de403a830 msgr2=0x7f2de40e20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.625+0000 7f2df2e8d640 1 -- 172.21.15.17:0/3140309637 shutdown_connections 2024-03-16T10:11:04.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.625+0000 7f2df2e8d640 1 -- 172.21.15.17:0/3140309637 wait complete. 2024-03-16T10:11:04.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-03-16T10:11:04.643 DEBUG:teuthology.parallel:result is None 2024-03-16T10:11:04.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.677+0000 7f9a93053640 1 -- 172.21.15.17:0/815495100 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9a840d0450 con 0x7f9a840d0f80 2024-03-16T10:11:04.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.677+0000 7f9a82ffd640 1 -- 172.21.15.17:0/815495100 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9a8c16bdb0 con 0x7f9a840d0f80 2024-03-16T10:11:04.680 INFO:teuthology.orchestra.run.smithi017.stdout:55834574855 2024-03-16T10:11:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.677+0000 7f9a93053640 1 -- 172.21.15.17:0/815495100 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a70067730 msgr2=0x7f9a70069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.677+0000 7f9a93053640 1 --2- 172.21.15.17:0/815495100 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a70067730 0x7f9a70069bf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9a88006f50 tx=0x7f9a88006ea0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.677+0000 7f9a93053640 1 -- 172.21.15.17:0/815495100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a840d0f80 msgr2=0x7f9a84026370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.677+0000 7f9a93053640 1 --2- 172.21.15.17:0/815495100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a840d0f80 0x7f9a84026370 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c050860 tx=0x7f9a8c094dd0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.677+0000 7f9a93053640 1 -- 172.21.15.17:0/815495100 shutdown_connections 2024-03-16T10:11:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.677+0000 7f9a93053640 1 --2- 172.21.15.17:0/815495100 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a70067730 0x7f9a70069bf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.677+0000 7f9a93053640 1 --2- 172.21.15.17:0/815495100 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a840d0f80 0x7f9a84026370 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.677+0000 7f9a93053640 1 --2- 172.21.15.17:0/815495100 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a8401f1a0 0x7f9a840289e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.677+0000 7f9a93053640 1 --2- 172.21.15.17:0/815495100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a8400c360 0x7f9a84027890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.677+0000 7f9a93053640 1 -- 172.21.15.17:0/815495100 >> 172.21.15.17:0/815495100 conn(0x7f9a8400b5e0 msgr2=0x7f9a8403acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.677+0000 7f9a93053640 1 -- 172.21.15.17:0/815495100 shutdown_connections 2024-03-16T10:11:04.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.677+0000 7f9a93053640 1 -- 172.21.15.17:0/815495100 wait complete. 2024-03-16T10:11:04.697 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2024-03-16T10:11:04.697 DEBUG:teuthology.parallel:result is None 2024-03-16T10:11:04.697 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-16T10:11:04.697 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:11:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69f04b4640 1 Processor -- start 2024-03-16T10:11:04.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69f04b4640 1 -- start start 2024-03-16T10:11:04.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69f04b4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69e8067190 0x7f69e813f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69f04b4640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f69e813f610 0x7f69e814fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69f04b4640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e80b4c20 con 0x7f69e806e000 2024-03-16T10:11:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69f04b4640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f69e809bd00 con 0x7f69e813f610 2024-03-16T10:11:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69f04b4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f69e809bf50 con 0x7f69e8067190 2024-03-16T10:11:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69eea51640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f69e813f610 0x7f69e814fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69eea51640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f69e813f610 0x7f69e814fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.17:40226/0 (socket says 172.21.15.17:40226) 2024-03-16T10:11:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69eea51640 1 -- 172.21.15.17:0/2947152978 learned_addr learned my addr 172.21.15.17:0/2947152978 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69eda4f640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f69e806e000 0x7f69e8148750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.17:58696/0 (socket says 172.21.15.17:58696) 2024-03-16T10:11:04.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69ee250640 1 --2- 172.21.15.17:0/2947152978 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69e8067190 0x7f69e813f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69eea51640 1 -- 172.21.15.17:0/2947152978 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69e8067190 msgr2=0x7f69e813f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:04.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69eea51640 1 --2- 172.21.15.17:0/2947152978 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69e8067190 0x7f69e813f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69eea51640 1 -- 172.21.15.17:0/2947152978 >> v1:172.21.15.174:6789/0 conn(0x7f69e806e000 legacy=0x7f69e8148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69eea51640 1 -- 172.21.15.17:0/2947152978 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e80b4420 con 0x7f69e813f610 2024-03-16T10:11:04.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69ed24e640 1 -- 172.21.15.17:0/2947152978 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3750998844 0 0) 0x7f69e80b4c20 con 0x7f69e806e000 2024-03-16T10:11:04.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.861+0000 7f69ee250640 1 --2- 172.21.15.17:0/2947152978 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69e8067190 0x7f69e813f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:11:04.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69eea51640 1 --2- 172.21.15.17:0/2947152978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69e813f610 0x7f69e814fae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009e50 tx=0x7f69e001c9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=155bbbe89daca244 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69ed24e640 1 -- 172.21.15.17:0/2947152978 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e00a56f0 con 0x7f69e813f610 2024-03-16T10:11:04.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69ed24e640 1 -- 172.21.15.17:0/2947152978 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e00a5890 con 0x7f69e813f610 2024-03-16T10:11:04.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69ed24e640 1 -- 172.21.15.17:0/2947152978 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e00a5c60 con 0x7f69e813f610 2024-03-16T10:11:04.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 -- 172.21.15.17:0/2947152978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69e813f610 msgr2=0x7f69e814fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 --2- 172.21.15.17:0/2947152978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69e813f610 0x7f69e814fae0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009e50 tx=0x7f69e001c9a0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 -- 172.21.15.17:0/2947152978 shutdown_connections 2024-03-16T10:11:04.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 --2- 172.21.15.17:0/2947152978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69e813f610 0x7f69e814fae0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 --2- 172.21.15.17:0/2947152978 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69e8067190 0x7f69e813f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 -- 172.21.15.17:0/2947152978 >> 172.21.15.17:0/2947152978 conn(0x7f69e80b5e10 msgr2=0x7f69e80b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:04.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 -- 172.21.15.17:0/2947152978 shutdown_connections 2024-03-16T10:11:04.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 -- 172.21.15.17:0/2947152978 wait complete. 2024-03-16T10:11:04.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 Processor -- start 2024-03-16T10:11:04.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 -- start start 2024-03-16T10:11:04.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69e8067190 0x7f69e813dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e806e000 0x7f69e81e0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69e813f610 0x7f69e8135510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f69e8148e60 con 0x7f69e806e000 2024-03-16T10:11:04.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69e80b6c20 con 0x7f69e8067190 2024-03-16T10:11:04.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69f04b4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f69e8148fe0 con 0x7f69e813f610 2024-03-16T10:11:04.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69eda4f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e806e000 0x7f69e81e0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69eda4f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e806e000 0x7f69e81e0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33866/0 (socket says 172.21.15.17:33866) 2024-03-16T10:11:04.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69eda4f640 1 -- 172.21.15.17:0/2600290194 learned_addr learned my addr 172.21.15.17:0/2600290194 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:04.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69ee250640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69e8067190 0x7f69e813dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69eea51640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69e813f610 0x7f69e8135510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69eda4f640 1 -- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69e813f610 msgr2=0x7f69e8135510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69eda4f640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69e813f610 0x7f69e8135510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69eda4f640 1 -- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69e8067190 msgr2=0x7f69e813dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:04.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69eda4f640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69e8067190 0x7f69e813dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:04.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69eda4f640 1 -- 172.21.15.17:0/2600290194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e8143870 con 0x7f69e806e000 2024-03-16T10:11:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69eea51640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69e813f610 0x7f69e8135510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.865+0000 7f69eda4f640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e806e000 0x7f69e81e0b20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f69d80011f0 tx=0x7f69d800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.869+0000 7f69deffd640 1 -- 172.21.15.17:0/2600290194 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8018070 con 0x7f69e806e000 2024-03-16T10:11:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.869+0000 7f69deffd640 1 -- 172.21.15.17:0/2600290194 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d80074e0 con 0x7f69e806e000 2024-03-16T10:11:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.869+0000 7f69f04b4640 1 -- 172.21.15.17:0/2600290194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e813cdf0 con 0x7f69e806e000 2024-03-16T10:11:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.869+0000 7f69deffd640 1 -- 172.21.15.17:0/2600290194 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8004a10 con 0x7f69e806e000 2024-03-16T10:11:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.869+0000 7f69f04b4640 1 -- 172.21.15.17:0/2600290194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e8143ae0 con 0x7f69e806e000 2024-03-16T10:11:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.869+0000 7f69f04b4640 1 -- 172.21.15.17:0/2600290194 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e806a130 con 0x7f69e806e000 2024-03-16T10:11:04.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.869+0000 7f69deffd640 1 -- 172.21.15.17:0/2600290194 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f69d801b070 con 0x7f69e806e000 2024-03-16T10:11:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.869+0000 7f69deffd640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f69cc067690 0x7f69cc069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.869+0000 7f69deffd640 1 -- 172.21.15.17:0/2600290194 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f69d8084b00 con 0x7f69e806e000 2024-03-16T10:11:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.869+0000 7f69ee250640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f69cc067690 0x7f69cc069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:04.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.869+0000 7f69ee250640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f69cc067690 0x7f69cc069b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009580 tx=0x7f69e4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:04.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:04.873+0000 7f69deffd640 1 -- 172.21.15.17:0/2600290194 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f69d8055120 con 0x7f69e806e000 2024-03-16T10:11:05.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.025+0000 7f69f04b4640 1 -- 172.21.15.17:0/2600290194 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f69e813cff0 con 0x7f69cc067690 2024-03-16T10:11:05.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.041+0000 7f69deffd640 1 -- 172.21.15.17:0/2600290194 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f69e813cff0 con 0x7f69cc067690 2024-03-16T10:11:05.049 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:05.050 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-16T10:11:05.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.045+0000 7f69f04b4640 1 -- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f69cc067690 msgr2=0x7f69cc069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.045+0000 7f69f04b4640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f69cc067690 0x7f69cc069b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f69e4009580 tx=0x7f69e4009190 comp rx=0 tx=0).stop 2024-03-16T10:11:05.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.045+0000 7f69f04b4640 1 -- 172.21.15.17:0/2600290194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e806e000 msgr2=0x7f69e81e0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.045+0000 7f69f04b4640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e806e000 0x7f69e81e0b20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f69d80011f0 tx=0x7f69d800c9b0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.045+0000 7f69f04b4640 1 -- 172.21.15.17:0/2600290194 shutdown_connections 2024-03-16T10:11:05.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.045+0000 7f69f04b4640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f69cc067690 0x7f69cc069b50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.045+0000 7f69f04b4640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f69e813f610 0x7f69e8135510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.045+0000 7f69f04b4640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69e806e000 0x7f69e81e0b20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.045+0000 7f69f04b4640 1 --2- 172.21.15.17:0/2600290194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69e8067190 0x7f69e813dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.045+0000 7f69f04b4640 1 -- 172.21.15.17:0/2600290194 >> 172.21.15.17:0/2600290194 conn(0x7f69e80b5e10 msgr2=0x7f69e809de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:05.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.045+0000 7f69f04b4640 1 -- 172.21.15.17:0/2600290194 shutdown_connections 2024-03-16T10:11:05.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.045+0000 7f69f04b4640 1 -- 172.21.15.17:0/2600290194 wait complete. 2024-03-16T10:11:05.072 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-16T10:11:04.016507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235732,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738988,"statfs":{"total":773094113280,"available":772852723712,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.932513"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830585+0000","last_change":"2024-03-16T10:10:58.137685+0000","last_active":"2024-03-16T10:10:59.830585+0000","last_peered":"2024-03-16T10:10:59.830585+0000","last_clean":"2024-03-16T10:10:59.830585+0000","last_became_active":"2024-03-16T10:10:58.136911+0000","last_became_peered":"2024-03-16T10:10:58.136911+0000","last_unstale":"2024-03-16T10:10:59.830585+0000","last_undegraded":"2024-03-16T10:10:59.830585+0000","last_fullsized":"2024-03-16T10:10:59.830585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:31.763467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.117185+0000","last_change":"2024-03-16T10:10:58.131094+0000","last_active":"2024-03-16T10:10:59.117185+0000","last_peered":"2024-03-16T10:10:59.117185+0000","last_clean":"2024-03-16T10:10:59.117185+0000","last_became_active":"2024-03-16T10:10:58.130290+0000","last_became_peered":"2024-03-16T10:10:58.130290+0000","last_unstale":"2024-03-16T10:10:59.117185+0000","last_undegraded":"2024-03-16T10:10:59.117185+0000","last_fullsized":"2024-03-16T10:10:59.117185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465578+0000","last_change":"2024-03-16T10:10:58.143896+0000","last_active":"2024-03-16T10:10:59.465578+0000","last_peered":"2024-03-16T10:10:59.465578+0000","last_clean":"2024-03-16T10:10:59.465578+0000","last_became_active":"2024-03-16T10:10:58.143496+0000","last_became_peered":"2024-03-16T10:10:58.143496+0000","last_unstale":"2024-03-16T10:10:59.465578+0000","last_undegraded":"2024-03-16T10:10:59.465578+0000","last_fullsized":"2024-03-16T10:10:59.465578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:33:59.497661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123015+0000","last_change":"2024-03-16T10:10:58.150879+0000","last_active":"2024-03-16T10:10:59.123015+0000","last_peered":"2024-03-16T10:10:59.123015+0000","last_clean":"2024-03-16T10:10:59.123015+0000","last_became_active":"2024-03-16T10:10:58.149469+0000","last_became_peered":"2024-03-16T10:10:58.149469+0000","last_unstale":"2024-03-16T10:10:59.123015+0000","last_undegraded":"2024-03-16T10:10:59.123015+0000","last_fullsized":"2024-03-16T10:10:59.123015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:09:11.898626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830139+0000","last_change":"2024-03-16T10:10:58.135477+0000","last_active":"2024-03-16T10:10:59.830139+0000","last_peered":"2024-03-16T10:10:59.830139+0000","last_clean":"2024-03-16T10:10:59.830139+0000","last_became_active":"2024-03-16T10:10:58.134803+0000","last_became_peered":"2024-03-16T10:10:58.134803+0000","last_unstale":"2024-03-16T10:10:59.830139+0000","last_undegraded":"2024-03-16T10:10:59.830139+0000","last_fullsized":"2024-03-16T10:10:59.830139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:32.891269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123122+0000","last_change":"2024-03-16T10:10:58.148071+0000","last_active":"2024-03-16T10:10:59.123122+0000","last_peered":"2024-03-16T10:10:59.123122+0000","last_clean":"2024-03-16T10:10:59.123122+0000","last_became_active":"2024-03-16T10:10:58.147304+0000","last_became_peered":"2024-03-16T10:10:58.147304+0000","last_unstale":"2024-03-16T10:10:59.123122+0000","last_undegraded":"2024-03-16T10:10:59.123122+0000","last_fullsized":"2024-03-16T10:10:59.123122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:46:09.470489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829729+0000","last_change":"2024-03-16T10:10:58.129866+0000","last_active":"2024-03-16T10:10:59.829729+0000","last_peered":"2024-03-16T10:10:59.829729+0000","last_clean":"2024-03-16T10:10:59.829729+0000","last_became_active":"2024-03-16T10:10:58.129125+0000","last_became_peered":"2024-03-16T10:10:58.129125+0000","last_unstale":"2024-03-16T10:10:59.829729+0000","last_undegraded":"2024-03-16T10:10:59.829729+0000","last_fullsized":"2024-03-16T10:10:59.829729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:25:29.088835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830012+0000","last_change":"2024-03-16T10:10:58.169409+0000","last_active":"2024-03-16T10:10:59.830012+0000","last_peered":"2024-03-16T10:10:59.830012+0000","last_clean":"2024-03-16T10:10:59.830012+0000","last_became_active":"2024-03-16T10:10:58.169086+0000","last_became_peered":"2024-03-16T10:10:58.169086+0000","last_unstale":"2024-03-16T10:10:59.830012+0000","last_undegraded":"2024-03-16T10:10:59.830012+0000","last_fullsized":"2024-03-16T10:10:59.830012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830943+0000","last_change":"2024-03-16T10:10:58.175200+0000","last_active":"2024-03-16T10:10:59.830943+0000","last_peered":"2024-03-16T10:10:59.830943+0000","last_clean":"2024-03-16T10:10:59.830943+0000","last_became_active":"2024-03-16T10:10:58.174581+0000","last_became_peered":"2024-03-16T10:10:58.174581+0000","last_unstale":"2024-03-16T10:10:59.830943+0000","last_undegraded":"2024-03-16T10:10:59.830943+0000","last_fullsized":"2024-03-16T10:10:59.830943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:39.149098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120166+0000","last_change":"2024-03-16T10:10:58.141861+0000","last_active":"2024-03-16T10:10:59.120166+0000","last_peered":"2024-03-16T10:10:59.120166+0000","last_clean":"2024-03-16T10:10:59.120166+0000","last_became_active":"2024-03-16T10:10:58.139720+0000","last_became_peered":"2024-03-16T10:10:58.139720+0000","last_unstale":"2024-03-16T10:10:59.120166+0000","last_undegraded":"2024-03-16T10:10:59.120166+0000","last_fullsized":"2024-03-16T10:10:59.120166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119960+0000","last_change":"2024-03-16T10:10:58.134873+0000","last_active":"2024-03-16T10:10:59.119960+0000","last_peered":"2024-03-16T10:10:59.119960+0000","last_clean":"2024-03-16T10:10:59.119960+0000","last_became_active":"2024-03-16T10:10:58.134062+0000","last_became_peered":"2024-03-16T10:10:58.134062+0000","last_unstale":"2024-03-16T10:10:59.119960+0000","last_undegraded":"2024-03-16T10:10:59.119960+0000","last_fullsized":"2024-03-16T10:10:59.119960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121413+0000","last_change":"2024-03-16T10:10:58.132580+0000","last_active":"2024-03-16T10:10:59.121413+0000","last_peered":"2024-03-16T10:10:59.121413+0000","last_clean":"2024-03-16T10:10:59.121413+0000","last_became_active":"2024-03-16T10:10:58.130316+0000","last_became_peered":"2024-03-16T10:10:58.130316+0000","last_unstale":"2024-03-16T10:10:59.121413+0000","last_undegraded":"2024-03-16T10:10:59.121413+0000","last_fullsized":"2024-03-16T10:10:59.121413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:18.684392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.831259+0000","last_change":"2024-03-16T10:10:58.129802+0000","last_active":"2024-03-16T10:10:59.831259+0000","last_peered":"2024-03-16T10:10:59.831259+0000","last_clean":"2024-03-16T10:10:59.831259+0000","last_became_active":"2024-03-16T10:10:58.129060+0000","last_became_peered":"2024-03-16T10:10:58.129060+0000","last_unstale":"2024-03-16T10:10:59.831259+0000","last_undegraded":"2024-03-16T10:10:59.831259+0000","last_fullsized":"2024-03-16T10:10:59.831259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:09:11.898626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466226+0000","last_change":"2024-03-16T10:10:58.146456+0000","last_active":"2024-03-16T10:10:59.466226+0000","last_peered":"2024-03-16T10:10:59.466226+0000","last_clean":"2024-03-16T10:10:59.466226+0000","last_became_active":"2024-03-16T10:10:58.146121+0000","last_became_peered":"2024-03-16T10:10:58.146121+0000","last_unstale":"2024-03-16T10:10:59.466226+0000","last_undegraded":"2024-03-16T10:10:59.466226+0000","last_fullsized":"2024-03-16T10:10:59.466226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.115785+0000","last_change":"2024-03-16T10:10:58.128942+0000","last_active":"2024-03-16T10:10:59.115785+0000","last_peered":"2024-03-16T10:10:59.115785+0000","last_clean":"2024-03-16T10:10:59.115785+0000","last_became_active":"2024-03-16T10:10:58.128181+0000","last_became_peered":"2024-03-16T10:10:58.128181+0000","last_unstale":"2024-03-16T10:10:59.115785+0000","last_undegraded":"2024-03-16T10:10:59.115785+0000","last_fullsized":"2024-03-16T10:10:59.115785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122518+0000","last_change":"2024-03-16T10:10:58.149792+0000","last_active":"2024-03-16T10:10:59.122518+0000","last_peered":"2024-03-16T10:10:59.122518+0000","last_clean":"2024-03-16T10:10:59.122518+0000","last_became_active":"2024-03-16T10:10:58.148809+0000","last_became_peered":"2024-03-16T10:10:58.148809+0000","last_unstale":"2024-03-16T10:10:59.122518+0000","last_undegraded":"2024-03-16T10:10:59.122518+0000","last_fullsized":"2024-03-16T10:10:59.122518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:01:30.933303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.124057+0000","last_change":"2024-03-16T10:10:58.159388+0000","last_active":"2024-03-16T10:10:59.124057+0000","last_peered":"2024-03-16T10:10:59.124057+0000","last_clean":"2024-03-16T10:10:59.124057+0000","last_became_active":"2024-03-16T10:10:58.151577+0000","last_became_peered":"2024-03-16T10:10:58.151577+0000","last_unstale":"2024-03-16T10:10:59.124057+0000","last_undegraded":"2024-03-16T10:10:59.124057+0000","last_fullsized":"2024-03-16T10:10:59.124057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:12.960572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829702+0000","last_change":"2024-03-16T10:10:58.141141+0000","last_active":"2024-03-16T10:10:59.829702+0000","last_peered":"2024-03-16T10:10:59.829702+0000","last_clean":"2024-03-16T10:10:59.829702+0000","last_became_active":"2024-03-16T10:10:58.139690+0000","last_became_peered":"2024-03-16T10:10:58.139690+0000","last_unstale":"2024-03-16T10:10:59.829702+0000","last_undegraded":"2024-03-16T10:10:59.829702+0000","last_fullsized":"2024-03-16T10:10:59.829702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:56:48.783143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830812+0000","last_change":"2024-03-16T10:10:58.138018+0000","last_active":"2024-03-16T10:10:59.830812+0000","last_peered":"2024-03-16T10:10:59.830812+0000","last_clean":"2024-03-16T10:10:59.830812+0000","last_became_active":"2024-03-16T10:10:58.137216+0000","last_became_peered":"2024-03-16T10:10:58.137216+0000","last_unstale":"2024-03-16T10:10:59.830812+0000","last_undegraded":"2024-03-16T10:10:59.830812+0000","last_fullsized":"2024-03-16T10:10:59.830812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:08.743197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120346+0000","last_change":"2024-03-16T10:10:58.159417+0000","last_active":"2024-03-16T10:10:59.120346+0000","last_peered":"2024-03-16T10:10:59.120346+0000","last_clean":"2024-03-16T10:10:59.120346+0000","last_became_active":"2024-03-16T10:10:58.158720+0000","last_became_peered":"2024-03-16T10:10:58.158720+0000","last_unstale":"2024-03-16T10:10:59.120346+0000","last_undegraded":"2024-03-16T10:10:59.120346+0000","last_fullsized":"2024-03-16T10:10:59.120346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465780+0000","last_change":"2024-03-16T10:10:58.140446+0000","last_active":"2024-03-16T10:10:59.465780+0000","last_peered":"2024-03-16T10:10:59.465780+0000","last_clean":"2024-03-16T10:10:59.465780+0000","last_became_active":"2024-03-16T10:10:58.139239+0000","last_became_peered":"2024-03-16T10:10:58.139239+0000","last_unstale":"2024-03-16T10:10:59.465780+0000","last_undegraded":"2024-03-16T10:10:59.465780+0000","last_fullsized":"2024-03-16T10:10:59.465780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:18:51.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123714+0000","last_change":"2024-03-16T10:10:58.168915+0000","last_active":"2024-03-16T10:10:59.123714+0000","last_peered":"2024-03-16T10:10:59.123714+0000","last_clean":"2024-03-16T10:10:59.123714+0000","last_became_active":"2024-03-16T10:10:58.159348+0000","last_became_peered":"2024-03-16T10:10:58.159348+0000","last_unstale":"2024-03-16T10:10:59.123714+0000","last_undegraded":"2024-03-16T10:10:59.123714+0000","last_fullsized":"2024-03-16T10:10:59.123714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:39.149098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829952+0000","last_change":"2024-03-16T10:10:58.166358+0000","last_active":"2024-03-16T10:10:59.829952+0000","last_peered":"2024-03-16T10:10:59.829952+0000","last_clean":"2024-03-16T10:10:59.829952+0000","last_became_active":"2024-03-16T10:10:58.166047+0000","last_became_peered":"2024-03-16T10:10:58.166047+0000","last_unstale":"2024-03-16T10:10:59.829952+0000","last_undegraded":"2024-03-16T10:10:59.829952+0000","last_fullsized":"2024-03-16T10:10:59.829952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:18:51.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122656+0000","last_change":"2024-03-16T10:10:58.165962+0000","last_active":"2024-03-16T10:10:59.122656+0000","last_peered":"2024-03-16T10:10:59.122656+0000","last_clean":"2024-03-16T10:10:59.122656+0000","last_became_active":"2024-03-16T10:10:58.164489+0000","last_became_peered":"2024-03-16T10:10:58.164489+0000","last_unstale":"2024-03-16T10:10:59.122656+0000","last_undegraded":"2024-03-16T10:10:59.122656+0000","last_fullsized":"2024-03-16T10:10:59.122656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:25:29.088835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830935+0000","last_change":"2024-03-16T10:10:58.156904+0000","last_active":"2024-03-16T10:10:59.830935+0000","last_peered":"2024-03-16T10:10:59.830935+0000","last_clean":"2024-03-16T10:10:59.830935+0000","last_became_active":"2024-03-16T10:10:58.156570+0000","last_became_peered":"2024-03-16T10:10:58.156570+0000","last_unstale":"2024-03-16T10:10:59.830935+0000","last_undegraded":"2024-03-16T10:10:59.830935+0000","last_fullsized":"2024-03-16T10:10:59.830935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:01:30.933303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120153+0000","last_change":"2024-03-16T10:10:58.139427+0000","last_active":"2024-03-16T10:10:59.120153+0000","last_peered":"2024-03-16T10:10:59.120153+0000","last_clean":"2024-03-16T10:10:59.120153+0000","last_became_active":"2024-03-16T10:10:58.138492+0000","last_became_peered":"2024-03-16T10:10:58.138492+0000","last_unstale":"2024-03-16T10:10:59.120153+0000","last_undegraded":"2024-03-16T10:10:59.120153+0000","last_fullsized":"2024-03-16T10:10:59.120153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116872+0000","last_change":"2024-03-16T10:10:58.141336+0000","last_active":"2024-03-16T10:10:59.116872+0000","last_peered":"2024-03-16T10:10:59.116872+0000","last_clean":"2024-03-16T10:10:59.116872+0000","last_became_active":"2024-03-16T10:10:58.139760+0000","last_became_peered":"2024-03-16T10:10:58.139760+0000","last_unstale":"2024-03-16T10:10:59.116872+0000","last_undegraded":"2024-03-16T10:10:59.116872+0000","last_fullsized":"2024-03-16T10:10:59.116872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830701+0000","last_change":"2024-03-16T10:10:58.166524+0000","last_active":"2024-03-16T10:10:59.830701+0000","last_peered":"2024-03-16T10:10:59.830701+0000","last_clean":"2024-03-16T10:10:59.830701+0000","last_became_active":"2024-03-16T10:10:58.166197+0000","last_became_peered":"2024-03-16T10:10:58.166197+0000","last_unstale":"2024-03-16T10:10:59.830701+0000","last_undegraded":"2024-03-16T10:10:59.830701+0000","last_fullsized":"2024-03-16T10:10:59.830701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466121+0000","last_change":"2024-03-16T10:10:58.138295+0000","last_active":"2024-03-16T10:10:59.466121+0000","last_peered":"2024-03-16T10:10:59.466121+0000","last_clean":"2024-03-16T10:10:59.466121+0000","last_became_active":"2024-03-16T10:10:58.136934+0000","last_became_peered":"2024-03-16T10:10:58.136934+0000","last_unstale":"2024-03-16T10:10:59.466121+0000","last_undegraded":"2024-03-16T10:10:59.466121+0000","last_fullsized":"2024-03-16T10:10:59.466121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123600+0000","last_change":"2024-03-16T10:10:58.167036+0000","last_active":"2024-03-16T10:10:59.123600+0000","last_peered":"2024-03-16T10:10:59.123600+0000","last_clean":"2024-03-16T10:10:59.123600+0000","last_became_active":"2024-03-16T10:10:58.150783+0000","last_became_peered":"2024-03-16T10:10:58.150783+0000","last_unstale":"2024-03-16T10:10:59.123600+0000","last_undegraded":"2024-03-16T10:10:59.123600+0000","last_fullsized":"2024-03-16T10:10:59.123600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:13.329157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829447+0000","last_change":"2024-03-16T10:10:58.135886+0000","last_active":"2024-03-16T10:10:59.829447+0000","last_peered":"2024-03-16T10:10:59.829447+0000","last_clean":"2024-03-16T10:10:59.829447+0000","last_became_active":"2024-03-16T10:10:58.135152+0000","last_became_peered":"2024-03-16T10:10:58.135152+0000","last_unstale":"2024-03-16T10:10:59.829447+0000","last_undegraded":"2024-03-16T10:10:59.829447+0000","last_fullsized":"2024-03-16T10:10:59.829447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:33:04.565468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.118840+0000","last_change":"2024-03-16T10:10:58.124028+0000","last_active":"2024-03-16T10:10:59.118840+0000","last_peered":"2024-03-16T10:10:59.118840+0000","last_clean":"2024-03-16T10:10:59.118840+0000","last_became_active":"2024-03-16T10:10:58.123127+0000","last_became_peered":"2024-03-16T10:10:58.123127+0000","last_unstale":"2024-03-16T10:10:59.118840+0000","last_undegraded":"2024-03-16T10:10:59.118840+0000","last_fullsized":"2024-03-16T10:10:59.118840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:18:51.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119320+0000","last_change":"2024-03-16T10:10:58.141873+0000","last_active":"2024-03-16T10:10:59.119320+0000","last_peered":"2024-03-16T10:10:59.119320+0000","last_clean":"2024-03-16T10:10:59.119320+0000","last_became_active":"2024-03-16T10:10:58.134577+0000","last_became_peered":"2024-03-16T10:10:58.134577+0000","last_unstale":"2024-03-16T10:10:59.119320+0000","last_undegraded":"2024-03-16T10:10:59.119320+0000","last_fullsized":"2024-03-16T10:10:59.119320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:36.702176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.831022+0000","last_change":"2024-03-16T10:10:57.116491+0000","last_active":"2024-03-16T10:10:59.831022+0000","last_peered":"2024-03-16T10:10:59.831022+0000","last_clean":"2024-03-16T10:10:59.831022+0000","last_became_active":"2024-03-16T10:10:57.116165+0000","last_became_peered":"2024-03-16T10:10:57.116165+0000","last_unstale":"2024-03-16T10:10:59.831022+0000","last_undegraded":"2024-03-16T10:10:59.831022+0000","last_fullsized":"2024-03-16T10:10:59.831022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:32:37.635494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466258+0000","last_change":"2024-03-16T10:10:58.158940+0000","last_active":"2024-03-16T10:10:59.466258+0000","last_peered":"2024-03-16T10:10:59.466258+0000","last_clean":"2024-03-16T10:10:59.466258+0000","last_became_active":"2024-03-16T10:10:58.158624+0000","last_became_peered":"2024-03-16T10:10:58.158624+0000","last_unstale":"2024-03-16T10:10:59.466258+0000","last_undegraded":"2024-03-16T10:10:59.466258+0000","last_fullsized":"2024-03-16T10:10:59.466258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120492+0000","last_change":"2024-03-16T10:10:57.103389+0000","last_active":"2024-03-16T10:10:59.120492+0000","last_peered":"2024-03-16T10:10:59.120492+0000","last_clean":"2024-03-16T10:10:59.120492+0000","last_became_active":"2024-03-16T10:10:57.100522+0000","last_became_peered":"2024-03-16T10:10:57.100522+0000","last_unstale":"2024-03-16T10:10:59.120492+0000","last_undegraded":"2024-03-16T10:10:59.120492+0000","last_fullsized":"2024-03-16T10:10:59.120492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830305+0000","last_change":"2024-03-16T10:10:57.104411+0000","last_active":"2024-03-16T10:10:59.830305+0000","last_peered":"2024-03-16T10:10:59.830305+0000","last_clean":"2024-03-16T10:10:59.830305+0000","last_became_active":"2024-03-16T10:10:57.104091+0000","last_became_peered":"2024-03-16T10:10:57.104091+0000","last_unstale":"2024-03-16T10:10:59.830305+0000","last_undegraded":"2024-03-16T10:10:59.830305+0000","last_fullsized":"2024-03-16T10:10:59.830305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:46:56.525820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829910+0000","last_change":"2024-03-16T10:10:58.133972+0000","last_active":"2024-03-16T10:10:59.829910+0000","last_peered":"2024-03-16T10:10:59.829910+0000","last_clean":"2024-03-16T10:10:59.829910+0000","last_became_active":"2024-03-16T10:10:58.131756+0000","last_became_peered":"2024-03-16T10:10:58.131756+0000","last_unstale":"2024-03-16T10:10:59.829910+0000","last_undegraded":"2024-03-16T10:10:59.829910+0000","last_fullsized":"2024-03-16T10:10:59.829910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:33:59.497661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122858+0000","last_change":"2024-03-16T10:10:57.128184+0000","last_active":"2024-03-16T10:10:59.122858+0000","last_peered":"2024-03-16T10:10:59.122858+0000","last_clean":"2024-03-16T10:10:59.122858+0000","last_became_active":"2024-03-16T10:10:57.127626+0000","last_became_peered":"2024-03-16T10:10:57.127626+0000","last_unstale":"2024-03-16T10:10:59.122858+0000","last_undegraded":"2024-03-16T10:10:59.122858+0000","last_fullsized":"2024-03-16T10:10:59.122858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.115705+0000","last_change":"2024-03-16T10:10:58.133241+0000","last_active":"2024-03-16T10:10:59.115705+0000","last_peered":"2024-03-16T10:10:59.115705+0000","last_clean":"2024-03-16T10:10:59.115705+0000","last_became_active":"2024-03-16T10:10:58.132521+0000","last_became_peered":"2024-03-16T10:10:58.132521+0000","last_unstale":"2024-03-16T10:10:59.115705+0000","last_undegraded":"2024-03-16T10:10:59.115705+0000","last_fullsized":"2024-03-16T10:10:59.115705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120090+0000","last_change":"2024-03-16T10:10:57.134408+0000","last_active":"2024-03-16T10:10:59.120090+0000","last_peered":"2024-03-16T10:10:59.120090+0000","last_clean":"2024-03-16T10:10:59.120090+0000","last_became_active":"2024-03-16T10:10:57.133909+0000","last_became_peered":"2024-03-16T10:10:57.133909+0000","last_unstale":"2024-03-16T10:10:59.120090+0000","last_undegraded":"2024-03-16T10:10:59.120090+0000","last_fullsized":"2024-03-16T10:10:59.120090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:02.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123183+0000","last_change":"2024-03-16T10:10:58.158453+0000","last_active":"2024-03-16T10:10:59.123183+0000","last_peered":"2024-03-16T10:10:59.123183+0000","last_clean":"2024-03-16T10:10:59.123183+0000","last_became_active":"2024-03-16T10:10:58.151690+0000","last_became_peered":"2024-03-16T10:10:58.151690+0000","last_unstale":"2024-03-16T10:10:59.123183+0000","last_undegraded":"2024-03-16T10:10:59.123183+0000","last_fullsized":"2024-03-16T10:10:59.123183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:08.287161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123724+0000","last_change":"2024-03-16T10:10:57.137208+0000","last_active":"2024-03-16T10:10:59.123724+0000","last_peered":"2024-03-16T10:10:59.123724+0000","last_clean":"2024-03-16T10:10:59.123724+0000","last_became_active":"2024-03-16T10:10:57.136600+0000","last_became_peered":"2024-03-16T10:10:57.136600+0000","last_unstale":"2024-03-16T10:10:59.123724+0000","last_undegraded":"2024-03-16T10:10:59.123724+0000","last_fullsized":"2024-03-16T10:10:59.123724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:02.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466338+0000","last_change":"2024-03-16T10:10:58.166872+0000","last_active":"2024-03-16T10:10:59.466338+0000","last_peered":"2024-03-16T10:10:59.466338+0000","last_clean":"2024-03-16T10:10:59.466338+0000","last_became_active":"2024-03-16T10:10:58.166556+0000","last_became_peered":"2024-03-16T10:10:58.166556+0000","last_unstale":"2024-03-16T10:10:59.466338+0000","last_undegraded":"2024-03-16T10:10:59.466338+0000","last_fullsized":"2024-03-16T10:10:59.466338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116641+0000","last_change":"2024-03-16T10:10:57.131462+0000","last_active":"2024-03-16T10:10:59.116641+0000","last_peered":"2024-03-16T10:10:59.116641+0000","last_clean":"2024-03-16T10:10:59.116641+0000","last_became_active":"2024-03-16T10:10:57.131130+0000","last_became_peered":"2024-03-16T10:10:57.131130+0000","last_unstale":"2024-03-16T10:10:59.116641+0000","last_undegraded":"2024-03-16T10:10:59.116641+0000","last_fullsized":"2024-03-16T10:10:59.116641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:02.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121634+0000","last_change":"2024-03-16T10:10:58.144791+0000","last_active":"2024-03-16T10:10:59.121634+0000","last_peered":"2024-03-16T10:10:59.121634+0000","last_clean":"2024-03-16T10:10:59.121634+0000","last_became_active":"2024-03-16T10:10:58.144004+0000","last_became_peered":"2024-03-16T10:10:58.144004+0000","last_unstale":"2024-03-16T10:10:59.121634+0000","last_undegraded":"2024-03-16T10:10:59.121634+0000","last_fullsized":"2024-03-16T10:10:59.121634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:54.596181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120325+0000","last_change":"2024-03-16T10:10:57.101847+0000","last_active":"2024-03-16T10:10:59.120325+0000","last_peered":"2024-03-16T10:10:59.120325+0000","last_clean":"2024-03-16T10:10:59.120325+0000","last_became_active":"2024-03-16T10:10:57.100973+0000","last_became_peered":"2024-03-16T10:10:57.100973+0000","last_unstale":"2024-03-16T10:10:59.120325+0000","last_undegraded":"2024-03-16T10:10:59.120325+0000","last_fullsized":"2024-03-16T10:10:59.120325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830301+0000","last_change":"2024-03-16T10:10:58.141884+0000","last_active":"2024-03-16T10:10:59.830301+0000","last_peered":"2024-03-16T10:10:59.830301+0000","last_clean":"2024-03-16T10:10:59.830301+0000","last_became_active":"2024-03-16T10:10:58.140254+0000","last_became_peered":"2024-03-16T10:10:58.140254+0000","last_unstale":"2024-03-16T10:10:59.830301+0000","last_undegraded":"2024-03-16T10:10:59.830301+0000","last_fullsized":"2024-03-16T10:10:59.830301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:08.287161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466329+0000","last_change":"2024-03-16T10:10:57.117139+0000","last_active":"2024-03-16T10:10:59.466329+0000","last_peered":"2024-03-16T10:10:59.466329+0000","last_clean":"2024-03-16T10:10:59.466329+0000","last_became_active":"2024-03-16T10:10:57.111737+0000","last_became_peered":"2024-03-16T10:10:57.111737+0000","last_unstale":"2024-03-16T10:10:59.466329+0000","last_undegraded":"2024-03-16T10:10:59.466329+0000","last_fullsized":"2024-03-16T10:10:59.466329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:46:56.525820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465172+0000","last_change":"2024-03-16T10:10:58.140649+0000","last_active":"2024-03-16T10:10:59.465172+0000","last_peered":"2024-03-16T10:10:59.465172+0000","last_clean":"2024-03-16T10:10:59.465172+0000","last_became_active":"2024-03-16T10:10:58.139713+0000","last_became_peered":"2024-03-16T10:10:58.139713+0000","last_unstale":"2024-03-16T10:10:59.465172+0000","last_undegraded":"2024-03-16T10:10:59.465172+0000","last_fullsized":"2024-03-16T10:10:59.465172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:07:30.636156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123842+0000","last_change":"2024-03-16T10:10:57.138569+0000","last_active":"2024-03-16T10:10:59.123842+0000","last_peered":"2024-03-16T10:10:59.123842+0000","last_clean":"2024-03-16T10:10:59.123842+0000","last_became_active":"2024-03-16T10:10:57.136115+0000","last_became_peered":"2024-03-16T10:10:57.136115+0000","last_unstale":"2024-03-16T10:10:59.123842+0000","last_undegraded":"2024-03-16T10:10:59.123842+0000","last_fullsized":"2024-03-16T10:10:59.123842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830245+0000","last_change":"2024-03-16T10:10:58.130172+0000","last_active":"2024-03-16T10:10:59.830245+0000","last_peered":"2024-03-16T10:10:59.830245+0000","last_clean":"2024-03-16T10:10:59.830245+0000","last_became_active":"2024-03-16T10:10:58.120580+0000","last_became_peered":"2024-03-16T10:10:58.120580+0000","last_unstale":"2024-03-16T10:10:59.830245+0000","last_undegraded":"2024-03-16T10:10:59.830245+0000","last_fullsized":"2024-03-16T10:10:59.830245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:46:09.470489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120582+0000","last_change":"2024-03-16T10:10:57.104206+0000","last_active":"2024-03-16T10:10:59.120582+0000","last_peered":"2024-03-16T10:10:59.120582+0000","last_clean":"2024-03-16T10:10:59.120582+0000","last_became_active":"2024-03-16T10:10:57.100493+0000","last_became_peered":"2024-03-16T10:10:57.100493+0000","last_unstale":"2024-03-16T10:10:59.120582+0000","last_undegraded":"2024-03-16T10:10:59.120582+0000","last_fullsized":"2024-03-16T10:10:59.120582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:32:53.097656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119275+0000","last_change":"2024-03-16T10:10:58.123895+0000","last_active":"2024-03-16T10:10:59.119275+0000","last_peered":"2024-03-16T10:10:59.119275+0000","last_clean":"2024-03-16T10:10:59.119275+0000","last_became_active":"2024-03-16T10:10:58.122933+0000","last_became_peered":"2024-03-16T10:10:58.122933+0000","last_unstale":"2024-03-16T10:10:59.119275+0000","last_undegraded":"2024-03-16T10:10:59.119275+0000","last_fullsized":"2024-03-16T10:10:59.119275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829089+0000","last_change":"2024-03-16T10:10:57.105506+0000","last_active":"2024-03-16T10:10:59.829089+0000","last_peered":"2024-03-16T10:10:59.829089+0000","last_clean":"2024-03-16T10:10:59.829089+0000","last_became_active":"2024-03-16T10:10:57.104984+0000","last_became_peered":"2024-03-16T10:10:57.104984+0000","last_unstale":"2024-03-16T10:10:59.829089+0000","last_undegraded":"2024-03-16T10:10:59.829089+0000","last_fullsized":"2024-03-16T10:10:59.829089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:34:00.287180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829799+0000","last_change":"2024-03-16T10:10:58.129371+0000","last_active":"2024-03-16T10:10:59.829799+0000","last_peered":"2024-03-16T10:10:59.829799+0000","last_clean":"2024-03-16T10:10:59.829799+0000","last_became_active":"2024-03-16T10:10:58.121147+0000","last_became_peered":"2024-03-16T10:10:58.121147+0000","last_unstale":"2024-03-16T10:10:59.829799+0000","last_undegraded":"2024-03-16T10:10:59.829799+0000","last_fullsized":"2024-03-16T10:10:59.829799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:09:11.898626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121588+0000","last_change":"2024-03-16T10:10:57.093832+0000","last_active":"2024-03-16T10:10:59.121588+0000","last_peered":"2024-03-16T10:10:59.121588+0000","last_clean":"2024-03-16T10:10:59.121588+0000","last_became_active":"2024-03-16T10:10:57.093278+0000","last_became_peered":"2024-03-16T10:10:57.093278+0000","last_unstale":"2024-03-16T10:10:59.121588+0000","last_undegraded":"2024-03-16T10:10:59.121588+0000","last_fullsized":"2024-03-16T10:10:59.121588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:35:54.941983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830807+0000","last_change":"2024-03-16T10:10:58.169395+0000","last_active":"2024-03-16T10:10:59.830807+0000","last_peered":"2024-03-16T10:10:59.830807+0000","last_clean":"2024-03-16T10:10:59.830807+0000","last_became_active":"2024-03-16T10:10:58.168702+0000","last_became_peered":"2024-03-16T10:10:58.168702+0000","last_unstale":"2024-03-16T10:10:59.830807+0000","last_undegraded":"2024-03-16T10:10:59.830807+0000","last_fullsized":"2024-03-16T10:10:59.830807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:13.329157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829866+0000","last_change":"2024-03-16T10:10:57.105292+0000","last_active":"2024-03-16T10:10:59.829866+0000","last_peered":"2024-03-16T10:10:59.829866+0000","last_clean":"2024-03-16T10:10:59.829866+0000","last_became_active":"2024-03-16T10:10:57.104688+0000","last_became_peered":"2024-03-16T10:10:57.104688+0000","last_unstale":"2024-03-16T10:10:59.829866+0000","last_undegraded":"2024-03-16T10:10:59.829866+0000","last_fullsized":"2024-03-16T10:10:59.829866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830999+0000","last_change":"2024-03-16T10:10:58.170384+0000","last_active":"2024-03-16T10:10:59.830999+0000","last_peered":"2024-03-16T10:10:59.830999+0000","last_clean":"2024-03-16T10:10:59.830999+0000","last_became_active":"2024-03-16T10:10:58.169787+0000","last_became_peered":"2024-03-16T10:10:58.169787+0000","last_unstale":"2024-03-16T10:10:59.830999+0000","last_undegraded":"2024-03-16T10:10:59.830999+0000","last_fullsized":"2024-03-16T10:10:59.830999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:12.960572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116762+0000","last_change":"2024-03-16T10:10:57.111549+0000","last_active":"2024-03-16T10:10:59.116762+0000","last_peered":"2024-03-16T10:10:59.116762+0000","last_clean":"2024-03-16T10:10:59.116762+0000","last_became_active":"2024-03-16T10:10:57.111022+0000","last_became_peered":"2024-03-16T10:10:57.111022+0000","last_unstale":"2024-03-16T10:10:59.116762+0000","last_undegraded":"2024-03-16T10:10:59.116762+0000","last_fullsized":"2024-03-16T10:10:59.116762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:33:03.557374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123471+0000","last_change":"2024-03-16T10:10:58.150154+0000","last_active":"2024-03-16T10:10:59.123471+0000","last_peered":"2024-03-16T10:10:59.123471+0000","last_clean":"2024-03-16T10:10:59.123471+0000","last_became_active":"2024-03-16T10:10:58.149355+0000","last_became_peered":"2024-03-16T10:10:58.149355+0000","last_unstale":"2024-03-16T10:10:59.123471+0000","last_undegraded":"2024-03-16T10:10:59.123471+0000","last_fullsized":"2024-03-16T10:10:59.123471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:27:42.059927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829400+0000","last_change":"2024-03-16T10:10:57.129962+0000","last_active":"2024-03-16T10:10:59.829400+0000","last_peered":"2024-03-16T10:10:59.829400+0000","last_clean":"2024-03-16T10:10:59.829400+0000","last_became_active":"2024-03-16T10:10:57.129583+0000","last_became_peered":"2024-03-16T10:10:57.129583+0000","last_unstale":"2024-03-16T10:10:59.829400+0000","last_undegraded":"2024-03-16T10:10:59.829400+0000","last_fullsized":"2024-03-16T10:10:59.829400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:32:53.097656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830586+0000","last_change":"2024-03-16T10:10:58.140201+0000","last_active":"2024-03-16T10:10:59.830586+0000","last_peered":"2024-03-16T10:10:59.830586+0000","last_clean":"2024-03-16T10:10:59.830586+0000","last_became_active":"2024-03-16T10:10:58.139272+0000","last_became_peered":"2024-03-16T10:10:58.139272+0000","last_unstale":"2024-03-16T10:10:59.830586+0000","last_undegraded":"2024-03-16T10:10:59.830586+0000","last_fullsized":"2024-03-16T10:10:59.830586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:27:42.059927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829599+0000","last_change":"2024-03-16T10:10:48.248686+0000","last_active":"2024-03-16T10:10:59.829599+0000","last_peered":"2024-03-16T10:10:59.829599+0000","last_clean":"2024-03-16T10:10:59.829599+0000","last_became_active":"2024-03-16T10:10:48.243054+0000","last_became_peered":"2024-03-16T10:10:48.243054+0000","last_unstale":"2024-03-16T10:10:59.829599+0000","last_undegraded":"2024-03-16T10:10:59.829599+0000","last_fullsized":"2024-03-16T10:10:59.829599+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_clean_scrub_stamp":"2024-03-16T10:10:46.517938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:42.906473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122484+0000","last_change":"2024-03-16T10:10:57.123078+0000","last_active":"2024-03-16T10:10:59.122484+0000","last_peered":"2024-03-16T10:10:59.122484+0000","last_clean":"2024-03-16T10:10:59.122484+0000","last_became_active":"2024-03-16T10:10:57.121087+0000","last_became_peered":"2024-03-16T10:10:57.121087+0000","last_unstale":"2024-03-16T10:10:59.122484+0000","last_undegraded":"2024-03-16T10:10:59.122484+0000","last_fullsized":"2024-03-16T10:10:59.122484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830505+0000","last_change":"2024-03-16T10:10:58.171123+0000","last_active":"2024-03-16T10:10:59.830505+0000","last_peered":"2024-03-16T10:10:59.830505+0000","last_clean":"2024-03-16T10:10:59.830505+0000","last_became_active":"2024-03-16T10:10:58.170849+0000","last_became_peered":"2024-03-16T10:10:58.170849+0000","last_unstale":"2024-03-16T10:10:59.830505+0000","last_undegraded":"2024-03-16T10:10:59.830505+0000","last_fullsized":"2024-03-16T10:10:59.830505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829543+0000","last_change":"2024-03-16T10:10:57.124354+0000","last_active":"2024-03-16T10:10:59.829543+0000","last_peered":"2024-03-16T10:10:59.829543+0000","last_clean":"2024-03-16T10:10:59.829543+0000","last_became_active":"2024-03-16T10:10:57.123636+0000","last_became_peered":"2024-03-16T10:10:57.123636+0000","last_unstale":"2024-03-16T10:10:59.829543+0000","last_undegraded":"2024-03-16T10:10:59.829543+0000","last_fullsized":"2024-03-16T10:10:59.829543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465205+0000","last_change":"2024-03-16T10:10:58.158168+0000","last_active":"2024-03-16T10:10:59.465205+0000","last_peered":"2024-03-16T10:10:59.465205+0000","last_clean":"2024-03-16T10:10:59.465205+0000","last_became_active":"2024-03-16T10:10:58.157882+0000","last_became_peered":"2024-03-16T10:10:58.157882+0000","last_unstale":"2024-03-16T10:10:59.465205+0000","last_undegraded":"2024-03-16T10:10:59.465205+0000","last_fullsized":"2024-03-16T10:10:59.465205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:10:11.295606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829675+0000","last_change":"2024-03-16T10:10:57.104381+0000","last_active":"2024-03-16T10:10:59.829675+0000","last_peered":"2024-03-16T10:10:59.829675+0000","last_clean":"2024-03-16T10:10:59.829675+0000","last_became_active":"2024-03-16T10:10:57.103673+0000","last_became_peered":"2024-03-16T10:10:57.103673+0000","last_unstale":"2024-03-16T10:10:59.829675+0000","last_undegraded":"2024-03-16T10:10:59.829675+0000","last_fullsized":"2024-03-16T10:10:59.829675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:45:53.158955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.115477+0000","last_change":"2024-03-16T10:10:58.136233+0000","last_active":"2024-03-16T10:10:59.115477+0000","last_peered":"2024-03-16T10:10:59.115477+0000","last_clean":"2024-03-16T10:10:59.115477+0000","last_became_active":"2024-03-16T10:10:58.135287+0000","last_became_peered":"2024-03-16T10:10:58.135287+0000","last_unstale":"2024-03-16T10:10:59.115477+0000","last_undegraded":"2024-03-16T10:10:59.115477+0000","last_fullsized":"2024-03-16T10:10:59.115477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.124008+0000","last_change":"2024-03-16T10:10:57.132978+0000","last_active":"2024-03-16T10:10:59.124008+0000","last_peered":"2024-03-16T10:10:59.124008+0000","last_clean":"2024-03-16T10:10:59.124008+0000","last_became_active":"2024-03-16T10:10:57.132199+0000","last_became_peered":"2024-03-16T10:10:57.132199+0000","last_unstale":"2024-03-16T10:10:59.124008+0000","last_undegraded":"2024-03-16T10:10:59.124008+0000","last_fullsized":"2024-03-16T10:10:59.124008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:58.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121640+0000","last_change":"2024-03-16T10:10:58.131949+0000","last_active":"2024-03-16T10:10:59.121640+0000","last_peered":"2024-03-16T10:10:59.121640+0000","last_clean":"2024-03-16T10:10:59.121640+0000","last_became_active":"2024-03-16T10:10:58.131234+0000","last_became_peered":"2024-03-16T10:10:58.131234+0000","last_unstale":"2024-03-16T10:10:59.121640+0000","last_undegraded":"2024-03-16T10:10:59.121640+0000","last_fullsized":"2024-03-16T10:10:59.121640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:59.100283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116717+0000","last_change":"2024-03-16T10:10:57.115504+0000","last_active":"2024-03-16T10:10:59.116717+0000","last_peered":"2024-03-16T10:10:59.116717+0000","last_clean":"2024-03-16T10:10:59.116717+0000","last_became_active":"2024-03-16T10:10:57.115273+0000","last_became_peered":"2024-03-16T10:10:57.115273+0000","last_unstale":"2024-03-16T10:10:59.116717+0000","last_undegraded":"2024-03-16T10:10:59.116717+0000","last_fullsized":"2024-03-16T10:10:59.116717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:58.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123414+0000","last_change":"2024-03-16T10:10:58.150495+0000","last_active":"2024-03-16T10:10:59.123414+0000","last_peered":"2024-03-16T10:10:59.123414+0000","last_clean":"2024-03-16T10:10:59.123414+0000","last_became_active":"2024-03-16T10:10:58.149680+0000","last_became_peered":"2024-03-16T10:10:58.149680+0000","last_unstale":"2024-03-16T10:10:59.123414+0000","last_undegraded":"2024-03-16T10:10:59.123414+0000","last_fullsized":"2024-03-16T10:10:59.123414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:31.763467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830902+0000","last_change":"2024-03-16T10:10:57.128751+0000","last_active":"2024-03-16T10:10:59.830902+0000","last_peered":"2024-03-16T10:10:59.830902+0000","last_clean":"2024-03-16T10:10:59.830902+0000","last_became_active":"2024-03-16T10:10:57.127909+0000","last_became_peered":"2024-03-16T10:10:57.127909+0000","last_unstale":"2024-03-16T10:10:59.830902+0000","last_undegraded":"2024-03-16T10:10:59.830902+0000","last_fullsized":"2024-03-16T10:10:59.830902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:09:33.897012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116753+0000","last_change":"2024-03-16T10:10:58.134860+0000","last_active":"2024-03-16T10:10:59.116753+0000","last_peered":"2024-03-16T10:10:59.116753+0000","last_clean":"2024-03-16T10:10:59.116753+0000","last_became_active":"2024-03-16T10:10:58.133470+0000","last_became_peered":"2024-03-16T10:10:58.133470+0000","last_unstale":"2024-03-16T10:10:59.116753+0000","last_undegraded":"2024-03-16T10:10:59.116753+0000","last_fullsized":"2024-03-16T10:10:59.116753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.151156+0000","last_change":"2024-03-16T10:10:57.110493+0000","last_active":"2024-03-16T10:10:59.151156+0000","last_peered":"2024-03-16T10:10:59.151156+0000","last_clean":"2024-03-16T10:10:59.151156+0000","last_became_active":"2024-03-16T10:10:57.106136+0000","last_became_peered":"2024-03-16T10:10:57.106136+0000","last_unstale":"2024-03-16T10:10:59.151156+0000","last_undegraded":"2024-03-16T10:10:59.151156+0000","last_fullsized":"2024-03-16T10:10:59.151156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.831141+0000","last_change":"2024-03-16T10:10:58.140570+0000","last_active":"2024-03-16T10:10:59.831141+0000","last_peered":"2024-03-16T10:10:59.831141+0000","last_clean":"2024-03-16T10:10:59.831141+0000","last_became_active":"2024-03-16T10:10:58.139532+0000","last_became_peered":"2024-03-16T10:10:58.139532+0000","last_unstale":"2024-03-16T10:10:59.831141+0000","last_undegraded":"2024-03-16T10:10:59.831141+0000","last_fullsized":"2024-03-16T10:10:59.831141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:25:29.088835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120378+0000","last_change":"2024-03-16T10:10:57.120255+0000","last_active":"2024-03-16T10:10:59.120378+0000","last_peered":"2024-03-16T10:10:59.120378+0000","last_clean":"2024-03-16T10:10:59.120378+0000","last_became_active":"2024-03-16T10:10:57.119498+0000","last_became_peered":"2024-03-16T10:10:57.119498+0000","last_unstale":"2024-03-16T10:10:59.120378+0000","last_undegraded":"2024-03-16T10:10:59.120378+0000","last_fullsized":"2024-03-16T10:10:59.120378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465935+0000","last_change":"2024-03-16T10:10:58.129362+0000","last_active":"2024-03-16T10:10:59.465935+0000","last_peered":"2024-03-16T10:10:59.465935+0000","last_clean":"2024-03-16T10:10:59.465935+0000","last_became_active":"2024-03-16T10:10:58.128580+0000","last_became_peered":"2024-03-16T10:10:58.128580+0000","last_unstale":"2024-03-16T10:10:59.465935+0000","last_undegraded":"2024-03-16T10:10:59.465935+0000","last_fullsized":"2024-03-16T10:10:59.465935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829443+0000","last_change":"2024-03-16T10:10:57.118219+0000","last_active":"2024-03-16T10:10:59.829443+0000","last_peered":"2024-03-16T10:10:59.829443+0000","last_clean":"2024-03-16T10:10:59.829443+0000","last_became_active":"2024-03-16T10:10:57.117436+0000","last_became_peered":"2024-03-16T10:10:57.117436+0000","last_unstale":"2024-03-16T10:10:59.829443+0000","last_undegraded":"2024-03-16T10:10:59.829443+0000","last_fullsized":"2024-03-16T10:10:59.829443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119531+0000","last_change":"2024-03-16T10:10:58.143411+0000","last_active":"2024-03-16T10:10:59.119531+0000","last_peered":"2024-03-16T10:10:59.119531+0000","last_clean":"2024-03-16T10:10:59.119531+0000","last_became_active":"2024-03-16T10:10:58.142517+0000","last_became_peered":"2024-03-16T10:10:58.142517+0000","last_unstale":"2024-03-16T10:10:59.119531+0000","last_undegraded":"2024-03-16T10:10:59.119531+0000","last_fullsized":"2024-03-16T10:10:59.119531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:03.131047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116378+0000","last_change":"2024-03-16T10:10:57.116431+0000","last_active":"2024-03-16T10:10:59.116378+0000","last_peered":"2024-03-16T10:10:59.116378+0000","last_clean":"2024-03-16T10:10:59.116378+0000","last_became_active":"2024-03-16T10:10:57.115761+0000","last_became_peered":"2024-03-16T10:10:57.115761+0000","last_unstale":"2024-03-16T10:10:59.116378+0000","last_undegraded":"2024-03-16T10:10:59.116378+0000","last_fullsized":"2024-03-16T10:10:59.116378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830186+0000","last_change":"2024-03-16T10:10:58.137343+0000","last_active":"2024-03-16T10:10:59.830186+0000","last_peered":"2024-03-16T10:10:59.830186+0000","last_clean":"2024-03-16T10:10:59.830186+0000","last_became_active":"2024-03-16T10:10:58.136517+0000","last_became_peered":"2024-03-16T10:10:58.136517+0000","last_unstale":"2024-03-16T10:10:59.830186+0000","last_undegraded":"2024-03-16T10:10:59.830186+0000","last_fullsized":"2024-03-16T10:10:59.830186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116455+0000","last_change":"2024-03-16T10:10:57.100812+0000","last_active":"2024-03-16T10:10:59.116455+0000","last_peered":"2024-03-16T10:10:59.116455+0000","last_clean":"2024-03-16T10:10:59.116455+0000","last_became_active":"2024-03-16T10:10:57.100312+0000","last_became_peered":"2024-03-16T10:10:57.100312+0000","last_unstale":"2024-03-16T10:10:59.116455+0000","last_undegraded":"2024-03-16T10:10:59.116455+0000","last_fullsized":"2024-03-16T10:10:59.116455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830809+0000","last_change":"2024-03-16T10:10:58.139588+0000","last_active":"2024-03-16T10:10:59.830809+0000","last_peered":"2024-03-16T10:10:59.830809+0000","last_clean":"2024-03-16T10:10:59.830809+0000","last_became_active":"2024-03-16T10:10:58.136603+0000","last_became_peered":"2024-03-16T10:10:58.136603+0000","last_unstale":"2024-03-16T10:10:59.830809+0000","last_undegraded":"2024-03-16T10:10:59.830809+0000","last_fullsized":"2024-03-16T10:10:59.830809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830698+0000","last_change":"2024-03-16T10:10:57.127697+0000","last_active":"2024-03-16T10:10:59.830698+0000","last_peered":"2024-03-16T10:10:59.830698+0000","last_clean":"2024-03-16T10:10:59.830698+0000","last_became_active":"2024-03-16T10:10:57.126922+0000","last_became_peered":"2024-03-16T10:10:57.126922+0000","last_unstale":"2024-03-16T10:10:59.830698+0000","last_undegraded":"2024-03-16T10:10:59.830698+0000","last_fullsized":"2024-03-16T10:10:59.830698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:56:47.775049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120088+0000","last_change":"2024-03-16T10:10:58.162962+0000","last_active":"2024-03-16T10:10:59.120088+0000","last_peered":"2024-03-16T10:10:59.120088+0000","last_clean":"2024-03-16T10:10:59.120088+0000","last_became_active":"2024-03-16T10:10:58.161601+0000","last_became_peered":"2024-03-16T10:10:58.161601+0000","last_unstale":"2024-03-16T10:10:59.120088+0000","last_undegraded":"2024-03-16T10:10:59.120088+0000","last_fullsized":"2024-03-16T10:10:59.120088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123098+0000","last_change":"2024-03-16T10:10:58.160830+0000","last_active":"2024-03-16T10:10:59.123098+0000","last_peered":"2024-03-16T10:10:59.123098+0000","last_clean":"2024-03-16T10:10:59.123098+0000","last_became_active":"2024-03-16T10:10:58.160108+0000","last_became_peered":"2024-03-16T10:10:58.160108+0000","last_unstale":"2024-03-16T10:10:59.123098+0000","last_undegraded":"2024-03-16T10:10:59.123098+0000","last_fullsized":"2024-03-16T10:10:59.123098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:32.891269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830094+0000","last_change":"2024-03-16T10:10:57.138309+0000","last_active":"2024-03-16T10:10:59.830094+0000","last_peered":"2024-03-16T10:10:59.830094+0000","last_clean":"2024-03-16T10:10:59.830094+0000","last_became_active":"2024-03-16T10:10:57.137445+0000","last_became_peered":"2024-03-16T10:10:57.137445+0000","last_unstale":"2024-03-16T10:10:59.830094+0000","last_undegraded":"2024-03-16T10:10:59.830094+0000","last_fullsized":"2024-03-16T10:10:59.830094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:58.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123572+0000","last_change":"2024-03-16T10:10:58.160844+0000","last_active":"2024-03-16T10:10:59.123572+0000","last_peered":"2024-03-16T10:10:59.123572+0000","last_clean":"2024-03-16T10:10:59.123572+0000","last_became_active":"2024-03-16T10:10:58.160163+0000","last_became_peered":"2024-03-16T10:10:58.160163+0000","last_unstale":"2024-03-16T10:10:59.123572+0000","last_undegraded":"2024-03-16T10:10:59.123572+0000","last_fullsized":"2024-03-16T10:10:59.123572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:08.743197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830486+0000","last_change":"2024-03-16T10:10:57.126626+0000","last_active":"2024-03-16T10:10:59.830486+0000","last_peered":"2024-03-16T10:10:59.830486+0000","last_clean":"2024-03-16T10:10:59.830486+0000","last_became_active":"2024-03-16T10:10:57.125624+0000","last_became_peered":"2024-03-16T10:10:57.125624+0000","last_unstale":"2024-03-16T10:10:59.830486+0000","last_undegraded":"2024-03-16T10:10:59.830486+0000","last_fullsized":"2024-03-16T10:10:59.830486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:33:03.557374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465968+0000","last_change":"2024-03-16T10:10:58.149370+0000","last_active":"2024-03-16T10:10:59.465968+0000","last_peered":"2024-03-16T10:10:59.465968+0000","last_clean":"2024-03-16T10:10:59.465968+0000","last_became_active":"2024-03-16T10:10:58.149011+0000","last_became_peered":"2024-03-16T10:10:58.149011+0000","last_unstale":"2024-03-16T10:10:59.465968+0000","last_undegraded":"2024-03-16T10:10:59.465968+0000","last_fullsized":"2024-03-16T10:10:59.465968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830859+0000","last_change":"2024-03-16T10:10:57.114529+0000","last_active":"2024-03-16T10:10:59.830859+0000","last_peered":"2024-03-16T10:10:59.830859+0000","last_clean":"2024-03-16T10:10:59.830859+0000","last_became_active":"2024-03-16T10:10:57.113935+0000","last_became_peered":"2024-03-16T10:10:57.113935+0000","last_unstale":"2024-03-16T10:10:59.830859+0000","last_undegraded":"2024-03-16T10:10:59.830859+0000","last_fullsized":"2024-03-16T10:10:59.830859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:34:00.287180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830836+0000","last_change":"2024-03-16T10:10:58.138174+0000","last_active":"2024-03-16T10:10:59.830836+0000","last_peered":"2024-03-16T10:10:59.830836+0000","last_clean":"2024-03-16T10:10:59.830836+0000","last_became_active":"2024-03-16T10:10:58.136767+0000","last_became_peered":"2024-03-16T10:10:58.136767+0000","last_unstale":"2024-03-16T10:10:59.830836+0000","last_undegraded":"2024-03-16T10:10:59.830836+0000","last_fullsized":"2024-03-16T10:10:59.830836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466402+0000","last_change":"2024-03-16T10:10:57.124377+0000","last_active":"2024-03-16T10:10:59.466402+0000","last_peered":"2024-03-16T10:10:59.466402+0000","last_clean":"2024-03-16T10:10:59.466402+0000","last_became_active":"2024-03-16T10:10:57.123699+0000","last_became_peered":"2024-03-16T10:10:57.123699+0000","last_unstale":"2024-03-16T10:10:59.466402+0000","last_undegraded":"2024-03-16T10:10:59.466402+0000","last_fullsized":"2024-03-16T10:10:59.466402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:32:37.635494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:11:05.079 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:11:05.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.233+0000 7fe2d359e640 1 Processor -- start 2024-03-16T10:11:05.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.233+0000 7fe2d359e640 1 -- start start 2024-03-16T10:11:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d359e640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe2d413f680 0x7fe2d4145170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d359e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe2d409e440 0x7fe2d40741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d359e640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2d41488e0 con 0x7fe2d40b4510 2024-03-16T10:11:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d359e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe2d4148bc0 con 0x7fe2d409e440 2024-03-16T10:11:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d359e640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fe2d4148ea0 con 0x7fe2d413f680 2024-03-16T10:11:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d1d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe2d409e440 0x7fe2d40741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d1d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe2d409e440 0x7fe2d40741c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33868/0 (socket says 172.21.15.17:33868) 2024-03-16T10:11:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d1d9b640 1 -- 172.21.15.17:0/1830471242 learned_addr learned my addr 172.21.15.17:0/1830471242 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d259c640 1 --2- 172.21.15.17:0/1830471242 >> v2:172.21.15.174:3300/0 conn(0x7fe2d413f680 0x7fe2d4145170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d1d9b640 1 -- 172.21.15.17:0/1830471242 >> v2:172.21.15.174:3300/0 conn(0x7fe2d413f680 msgr2=0x7fe2d4145170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:05.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d1d9b640 1 --2- 172.21.15.17:0/1830471242 >> v2:172.21.15.174:3300/0 conn(0x7fe2d413f680 0x7fe2d4145170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d1d9b640 1 -- 172.21.15.17:0/1830471242 >> v1:172.21.15.174:6789/0 conn(0x7fe2d40b4510 legacy=0x7fe2d409d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:05.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d1d9b640 1 -- 172.21.15.17:0/1830471242 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d41456b0 con 0x7fe2d409e440 2024-03-16T10:11:05.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d1d9b640 1 --2- 172.21.15.17:0/1830471242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d409e440 0x7fe2d40741c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40b0580 tx=0x7fe2c801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0fc3d80bc7ce31 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:05.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d159a640 1 -- 172.21.15.17:0/1830471242 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80b0070 con 0x7fe2d409e440 2024-03-16T10:11:05.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d159a640 1 -- 172.21.15.17:0/1830471242 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c80a4e50 con 0x7fe2d409e440 2024-03-16T10:11:05.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.237+0000 7fe2d159a640 1 -- 172.21.15.17:0/1830471242 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80ab820 con 0x7fe2d409e440 2024-03-16T10:11:05.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 -- 172.21.15.17:0/1830471242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d409e440 msgr2=0x7fe2d40741c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 --2- 172.21.15.17:0/1830471242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d409e440 0x7fe2d40741c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40b0580 tx=0x7fe2c801c3c0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 -- 172.21.15.17:0/1830471242 shutdown_connections 2024-03-16T10:11:05.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 --2- 172.21.15.17:0/1830471242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d409e440 0x7fe2d40741c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 --2- 172.21.15.17:0/1830471242 >> v2:172.21.15.174:3300/0 conn(0x7fe2d413f680 0x7fe2d4145170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 -- 172.21.15.17:0/1830471242 >> 172.21.15.17:0/1830471242 conn(0x7fe2d406bc80 msgr2=0x7fe2d40b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:05.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 -- 172.21.15.17:0/1830471242 shutdown_connections 2024-03-16T10:11:05.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 -- 172.21.15.17:0/1830471242 wait complete. 2024-03-16T10:11:05.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 Processor -- start 2024-03-16T10:11:05.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 -- start start 2024-03-16T10:11:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d409e440 0x7fe2d4139060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d40b4510 0x7fe2d4139da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d413f680 0x7fe2d41de1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d4073850 con 0x7fe2d409e440 2024-03-16T10:11:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d409e440 0x7fe2d4139060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d259c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d409e440 0x7fe2d4139060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33872/0 (socket says 172.21.15.17:33872) 2024-03-16T10:11:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d259c640 1 -- 172.21.15.17:0/2820369031 learned_addr learned my addr 172.21.15.17:0/2820369031 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 -- 172.21.15.17:0/2820369031 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d40736d0 con 0x7fe2d40b4510 2024-03-16T10:11:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 -- 172.21.15.17:0/2820369031 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe2d40739d0 con 0x7fe2d413f680 2024-03-16T10:11:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d1d9b640 1 --2- 172.21.15.17:0/2820369031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d40b4510 0x7fe2d4139da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d2d9d640 1 --2- 172.21.15.17:0/2820369031 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d413f680 0x7fe2d41de1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d259c640 1 -- 172.21.15.17:0/2820369031 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d413f680 msgr2=0x7fe2d41de1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d259c640 1 --2- 172.21.15.17:0/2820369031 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d413f680 0x7fe2d41de1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d259c640 1 -- 172.21.15.17:0/2820369031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d40b4510 msgr2=0x7fe2d4139da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d259c640 1 --2- 172.21.15.17:0/2820369031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d40b4510 0x7fe2d4139da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d259c640 1 -- 172.21.15.17:0/2820369031 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d413a470 con 0x7fe2d409e440 2024-03-16T10:11:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d259c640 1 --2- 172.21.15.17:0/2820369031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d409e440 0x7fe2d4139060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400c820 tx=0x7fe2c400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2c2ffd640 1 -- 172.21.15.17:0/2820369031 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4015070 con 0x7fe2d409e440 2024-03-16T10:11:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2c2ffd640 1 -- 172.21.15.17:0/2820369031 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c4004020 con 0x7fe2d409e440 2024-03-16T10:11:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2c2ffd640 1 -- 172.21.15.17:0/2820369031 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4004450 con 0x7fe2d409e440 2024-03-16T10:11:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 -- 172.21.15.17:0/2820369031 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d413a750 con 0x7fe2d409e440 2024-03-16T10:11:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.241+0000 7fe2d359e640 1 -- 172.21.15.17:0/2820369031 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d4144f00 con 0x7fe2d409e440 2024-03-16T10:11:05.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.245+0000 7fe2d359e640 1 -- 172.21.15.17:0/2820369031 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2d4075520 con 0x7fe2d409e440 2024-03-16T10:11:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.245+0000 7fe2c2ffd640 1 -- 172.21.15.17:0/2820369031 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe2c4007830 con 0x7fe2d409e440 2024-03-16T10:11:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.245+0000 7fe2c2ffd640 1 --2- 172.21.15.17:0/2820369031 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe2b4067690 0x7fe2b4069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.245+0000 7fe2c2ffd640 1 -- 172.21.15.17:0/2820369031 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe2c40856f0 con 0x7fe2d409e440 2024-03-16T10:11:05.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.245+0000 7fe2d1d9b640 1 --2- 172.21.15.17:0/2820369031 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe2b4067690 0x7fe2b4069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.245+0000 7fe2d1d9b640 1 --2- 172.21.15.17:0/2820369031 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe2b4067690 0x7fe2b4069b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80ad000 tx=0x7fe2c8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:05.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.249+0000 7fe2c2ffd640 1 -- 172.21.15.17:0/2820369031 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe2c4056360 con 0x7fe2d409e440 2024-03-16T10:11:05.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.401+0000 7fe2d359e640 1 -- 172.21.15.17:0/2820369031 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe2d4070730 con 0x7fe2b4067690 2024-03-16T10:11:05.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.417+0000 7fe2c2ffd640 1 -- 172.21.15.17:0/2820369031 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7fe2d4070730 con 0x7fe2b4067690 2024-03-16T10:11:05.424 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:05.425 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-16T10:11:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.421+0000 7fe2d359e640 1 -- 172.21.15.17:0/2820369031 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe2b4067690 msgr2=0x7fe2b4069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.421+0000 7fe2d359e640 1 --2- 172.21.15.17:0/2820369031 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe2b4067690 0x7fe2b4069b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80ad000 tx=0x7fe2c8008040 comp rx=0 tx=0).stop 2024-03-16T10:11:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.421+0000 7fe2d359e640 1 -- 172.21.15.17:0/2820369031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d409e440 msgr2=0x7fe2d4139060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.421+0000 7fe2d359e640 1 --2- 172.21.15.17:0/2820369031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d409e440 0x7fe2d4139060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400c820 tx=0x7fe2c400cce0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.421+0000 7fe2d359e640 1 -- 172.21.15.17:0/2820369031 shutdown_connections 2024-03-16T10:11:05.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.421+0000 7fe2d359e640 1 --2- 172.21.15.17:0/2820369031 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe2b4067690 0x7fe2b4069b50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.421+0000 7fe2d359e640 1 --2- 172.21.15.17:0/2820369031 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe2d413f680 0x7fe2d41de1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.421+0000 7fe2d359e640 1 --2- 172.21.15.17:0/2820369031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d40b4510 0x7fe2d4139da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.421+0000 7fe2d359e640 1 --2- 172.21.15.17:0/2820369031 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe2d409e440 0x7fe2d4139060 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.421+0000 7fe2d359e640 1 -- 172.21.15.17:0/2820369031 >> 172.21.15.17:0/2820369031 conn(0x7fe2d406bc80 msgr2=0x7fe2d409d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.421+0000 7fe2d359e640 1 -- 172.21.15.17:0/2820369031 shutdown_connections 2024-03-16T10:11:05.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.421+0000 7fe2d359e640 1 -- 172.21.15.17:0/2820369031 wait complete. 2024-03-16T10:11:05.440 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-16T10:11:04.016507+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235732,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738988,"statfs":{"total":773094113280,"available":772852723712,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.932513"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830585+0000","last_change":"2024-03-16T10:10:58.137685+0000","last_active":"2024-03-16T10:10:59.830585+0000","last_peered":"2024-03-16T10:10:59.830585+0000","last_clean":"2024-03-16T10:10:59.830585+0000","last_became_active":"2024-03-16T10:10:58.136911+0000","last_became_peered":"2024-03-16T10:10:58.136911+0000","last_unstale":"2024-03-16T10:10:59.830585+0000","last_undegraded":"2024-03-16T10:10:59.830585+0000","last_fullsized":"2024-03-16T10:10:59.830585+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:31.763467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.117185+0000","last_change":"2024-03-16T10:10:58.131094+0000","last_active":"2024-03-16T10:10:59.117185+0000","last_peered":"2024-03-16T10:10:59.117185+0000","last_clean":"2024-03-16T10:10:59.117185+0000","last_became_active":"2024-03-16T10:10:58.130290+0000","last_became_peered":"2024-03-16T10:10:58.130290+0000","last_unstale":"2024-03-16T10:10:59.117185+0000","last_undegraded":"2024-03-16T10:10:59.117185+0000","last_fullsized":"2024-03-16T10:10:59.117185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465578+0000","last_change":"2024-03-16T10:10:58.143896+0000","last_active":"2024-03-16T10:10:59.465578+0000","last_peered":"2024-03-16T10:10:59.465578+0000","last_clean":"2024-03-16T10:10:59.465578+0000","last_became_active":"2024-03-16T10:10:58.143496+0000","last_became_peered":"2024-03-16T10:10:58.143496+0000","last_unstale":"2024-03-16T10:10:59.465578+0000","last_undegraded":"2024-03-16T10:10:59.465578+0000","last_fullsized":"2024-03-16T10:10:59.465578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:33:59.497661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123015+0000","last_change":"2024-03-16T10:10:58.150879+0000","last_active":"2024-03-16T10:10:59.123015+0000","last_peered":"2024-03-16T10:10:59.123015+0000","last_clean":"2024-03-16T10:10:59.123015+0000","last_became_active":"2024-03-16T10:10:58.149469+0000","last_became_peered":"2024-03-16T10:10:58.149469+0000","last_unstale":"2024-03-16T10:10:59.123015+0000","last_undegraded":"2024-03-16T10:10:59.123015+0000","last_fullsized":"2024-03-16T10:10:59.123015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:09:11.898626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830139+0000","last_change":"2024-03-16T10:10:58.135477+0000","last_active":"2024-03-16T10:10:59.830139+0000","last_peered":"2024-03-16T10:10:59.830139+0000","last_clean":"2024-03-16T10:10:59.830139+0000","last_became_active":"2024-03-16T10:10:58.134803+0000","last_became_peered":"2024-03-16T10:10:58.134803+0000","last_unstale":"2024-03-16T10:10:59.830139+0000","last_undegraded":"2024-03-16T10:10:59.830139+0000","last_fullsized":"2024-03-16T10:10:59.830139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:32.891269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123122+0000","last_change":"2024-03-16T10:10:58.148071+0000","last_active":"2024-03-16T10:10:59.123122+0000","last_peered":"2024-03-16T10:10:59.123122+0000","last_clean":"2024-03-16T10:10:59.123122+0000","last_became_active":"2024-03-16T10:10:58.147304+0000","last_became_peered":"2024-03-16T10:10:58.147304+0000","last_unstale":"2024-03-16T10:10:59.123122+0000","last_undegraded":"2024-03-16T10:10:59.123122+0000","last_fullsized":"2024-03-16T10:10:59.123122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:46:09.470489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829729+0000","last_change":"2024-03-16T10:10:58.129866+0000","last_active":"2024-03-16T10:10:59.829729+0000","last_peered":"2024-03-16T10:10:59.829729+0000","last_clean":"2024-03-16T10:10:59.829729+0000","last_became_active":"2024-03-16T10:10:58.129125+0000","last_became_peered":"2024-03-16T10:10:58.129125+0000","last_unstale":"2024-03-16T10:10:59.829729+0000","last_undegraded":"2024-03-16T10:10:59.829729+0000","last_fullsized":"2024-03-16T10:10:59.829729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:25:29.088835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830012+0000","last_change":"2024-03-16T10:10:58.169409+0000","last_active":"2024-03-16T10:10:59.830012+0000","last_peered":"2024-03-16T10:10:59.830012+0000","last_clean":"2024-03-16T10:10:59.830012+0000","last_became_active":"2024-03-16T10:10:58.169086+0000","last_became_peered":"2024-03-16T10:10:58.169086+0000","last_unstale":"2024-03-16T10:10:59.830012+0000","last_undegraded":"2024-03-16T10:10:59.830012+0000","last_fullsized":"2024-03-16T10:10:59.830012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830943+0000","last_change":"2024-03-16T10:10:58.175200+0000","last_active":"2024-03-16T10:10:59.830943+0000","last_peered":"2024-03-16T10:10:59.830943+0000","last_clean":"2024-03-16T10:10:59.830943+0000","last_became_active":"2024-03-16T10:10:58.174581+0000","last_became_peered":"2024-03-16T10:10:58.174581+0000","last_unstale":"2024-03-16T10:10:59.830943+0000","last_undegraded":"2024-03-16T10:10:59.830943+0000","last_fullsized":"2024-03-16T10:10:59.830943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:39.149098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120166+0000","last_change":"2024-03-16T10:10:58.141861+0000","last_active":"2024-03-16T10:10:59.120166+0000","last_peered":"2024-03-16T10:10:59.120166+0000","last_clean":"2024-03-16T10:10:59.120166+0000","last_became_active":"2024-03-16T10:10:58.139720+0000","last_became_peered":"2024-03-16T10:10:58.139720+0000","last_unstale":"2024-03-16T10:10:59.120166+0000","last_undegraded":"2024-03-16T10:10:59.120166+0000","last_fullsized":"2024-03-16T10:10:59.120166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119960+0000","last_change":"2024-03-16T10:10:58.134873+0000","last_active":"2024-03-16T10:10:59.119960+0000","last_peered":"2024-03-16T10:10:59.119960+0000","last_clean":"2024-03-16T10:10:59.119960+0000","last_became_active":"2024-03-16T10:10:58.134062+0000","last_became_peered":"2024-03-16T10:10:58.134062+0000","last_unstale":"2024-03-16T10:10:59.119960+0000","last_undegraded":"2024-03-16T10:10:59.119960+0000","last_fullsized":"2024-03-16T10:10:59.119960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121413+0000","last_change":"2024-03-16T10:10:58.132580+0000","last_active":"2024-03-16T10:10:59.121413+0000","last_peered":"2024-03-16T10:10:59.121413+0000","last_clean":"2024-03-16T10:10:59.121413+0000","last_became_active":"2024-03-16T10:10:58.130316+0000","last_became_peered":"2024-03-16T10:10:58.130316+0000","last_unstale":"2024-03-16T10:10:59.121413+0000","last_undegraded":"2024-03-16T10:10:59.121413+0000","last_fullsized":"2024-03-16T10:10:59.121413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:18.684392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.831259+0000","last_change":"2024-03-16T10:10:58.129802+0000","last_active":"2024-03-16T10:10:59.831259+0000","last_peered":"2024-03-16T10:10:59.831259+0000","last_clean":"2024-03-16T10:10:59.831259+0000","last_became_active":"2024-03-16T10:10:58.129060+0000","last_became_peered":"2024-03-16T10:10:58.129060+0000","last_unstale":"2024-03-16T10:10:59.831259+0000","last_undegraded":"2024-03-16T10:10:59.831259+0000","last_fullsized":"2024-03-16T10:10:59.831259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:09:11.898626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466226+0000","last_change":"2024-03-16T10:10:58.146456+0000","last_active":"2024-03-16T10:10:59.466226+0000","last_peered":"2024-03-16T10:10:59.466226+0000","last_clean":"2024-03-16T10:10:59.466226+0000","last_became_active":"2024-03-16T10:10:58.146121+0000","last_became_peered":"2024-03-16T10:10:58.146121+0000","last_unstale":"2024-03-16T10:10:59.466226+0000","last_undegraded":"2024-03-16T10:10:59.466226+0000","last_fullsized":"2024-03-16T10:10:59.466226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.115785+0000","last_change":"2024-03-16T10:10:58.128942+0000","last_active":"2024-03-16T10:10:59.115785+0000","last_peered":"2024-03-16T10:10:59.115785+0000","last_clean":"2024-03-16T10:10:59.115785+0000","last_became_active":"2024-03-16T10:10:58.128181+0000","last_became_peered":"2024-03-16T10:10:58.128181+0000","last_unstale":"2024-03-16T10:10:59.115785+0000","last_undegraded":"2024-03-16T10:10:59.115785+0000","last_fullsized":"2024-03-16T10:10:59.115785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122518+0000","last_change":"2024-03-16T10:10:58.149792+0000","last_active":"2024-03-16T10:10:59.122518+0000","last_peered":"2024-03-16T10:10:59.122518+0000","last_clean":"2024-03-16T10:10:59.122518+0000","last_became_active":"2024-03-16T10:10:58.148809+0000","last_became_peered":"2024-03-16T10:10:58.148809+0000","last_unstale":"2024-03-16T10:10:59.122518+0000","last_undegraded":"2024-03-16T10:10:59.122518+0000","last_fullsized":"2024-03-16T10:10:59.122518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:01:30.933303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.124057+0000","last_change":"2024-03-16T10:10:58.159388+0000","last_active":"2024-03-16T10:10:59.124057+0000","last_peered":"2024-03-16T10:10:59.124057+0000","last_clean":"2024-03-16T10:10:59.124057+0000","last_became_active":"2024-03-16T10:10:58.151577+0000","last_became_peered":"2024-03-16T10:10:58.151577+0000","last_unstale":"2024-03-16T10:10:59.124057+0000","last_undegraded":"2024-03-16T10:10:59.124057+0000","last_fullsized":"2024-03-16T10:10:59.124057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:12.960572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829702+0000","last_change":"2024-03-16T10:10:58.141141+0000","last_active":"2024-03-16T10:10:59.829702+0000","last_peered":"2024-03-16T10:10:59.829702+0000","last_clean":"2024-03-16T10:10:59.829702+0000","last_became_active":"2024-03-16T10:10:58.139690+0000","last_became_peered":"2024-03-16T10:10:58.139690+0000","last_unstale":"2024-03-16T10:10:59.829702+0000","last_undegraded":"2024-03-16T10:10:59.829702+0000","last_fullsized":"2024-03-16T10:10:59.829702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:56:48.783143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830812+0000","last_change":"2024-03-16T10:10:58.138018+0000","last_active":"2024-03-16T10:10:59.830812+0000","last_peered":"2024-03-16T10:10:59.830812+0000","last_clean":"2024-03-16T10:10:59.830812+0000","last_became_active":"2024-03-16T10:10:58.137216+0000","last_became_peered":"2024-03-16T10:10:58.137216+0000","last_unstale":"2024-03-16T10:10:59.830812+0000","last_undegraded":"2024-03-16T10:10:59.830812+0000","last_fullsized":"2024-03-16T10:10:59.830812+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:08.743197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120346+0000","last_change":"2024-03-16T10:10:58.159417+0000","last_active":"2024-03-16T10:10:59.120346+0000","last_peered":"2024-03-16T10:10:59.120346+0000","last_clean":"2024-03-16T10:10:59.120346+0000","last_became_active":"2024-03-16T10:10:58.158720+0000","last_became_peered":"2024-03-16T10:10:58.158720+0000","last_unstale":"2024-03-16T10:10:59.120346+0000","last_undegraded":"2024-03-16T10:10:59.120346+0000","last_fullsized":"2024-03-16T10:10:59.120346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465780+0000","last_change":"2024-03-16T10:10:58.140446+0000","last_active":"2024-03-16T10:10:59.465780+0000","last_peered":"2024-03-16T10:10:59.465780+0000","last_clean":"2024-03-16T10:10:59.465780+0000","last_became_active":"2024-03-16T10:10:58.139239+0000","last_became_peered":"2024-03-16T10:10:58.139239+0000","last_unstale":"2024-03-16T10:10:59.465780+0000","last_undegraded":"2024-03-16T10:10:59.465780+0000","last_fullsized":"2024-03-16T10:10:59.465780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:18:51.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123714+0000","last_change":"2024-03-16T10:10:58.168915+0000","last_active":"2024-03-16T10:10:59.123714+0000","last_peered":"2024-03-16T10:10:59.123714+0000","last_clean":"2024-03-16T10:10:59.123714+0000","last_became_active":"2024-03-16T10:10:58.159348+0000","last_became_peered":"2024-03-16T10:10:58.159348+0000","last_unstale":"2024-03-16T10:10:59.123714+0000","last_undegraded":"2024-03-16T10:10:59.123714+0000","last_fullsized":"2024-03-16T10:10:59.123714+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:02:39.149098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829952+0000","last_change":"2024-03-16T10:10:58.166358+0000","last_active":"2024-03-16T10:10:59.829952+0000","last_peered":"2024-03-16T10:10:59.829952+0000","last_clean":"2024-03-16T10:10:59.829952+0000","last_became_active":"2024-03-16T10:10:58.166047+0000","last_became_peered":"2024-03-16T10:10:58.166047+0000","last_unstale":"2024-03-16T10:10:59.829952+0000","last_undegraded":"2024-03-16T10:10:59.829952+0000","last_fullsized":"2024-03-16T10:10:59.829952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:18:51.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122656+0000","last_change":"2024-03-16T10:10:58.165962+0000","last_active":"2024-03-16T10:10:59.122656+0000","last_peered":"2024-03-16T10:10:59.122656+0000","last_clean":"2024-03-16T10:10:59.122656+0000","last_became_active":"2024-03-16T10:10:58.164489+0000","last_became_peered":"2024-03-16T10:10:58.164489+0000","last_unstale":"2024-03-16T10:10:59.122656+0000","last_undegraded":"2024-03-16T10:10:59.122656+0000","last_fullsized":"2024-03-16T10:10:59.122656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:25:29.088835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830935+0000","last_change":"2024-03-16T10:10:58.156904+0000","last_active":"2024-03-16T10:10:59.830935+0000","last_peered":"2024-03-16T10:10:59.830935+0000","last_clean":"2024-03-16T10:10:59.830935+0000","last_became_active":"2024-03-16T10:10:58.156570+0000","last_became_peered":"2024-03-16T10:10:58.156570+0000","last_unstale":"2024-03-16T10:10:59.830935+0000","last_undegraded":"2024-03-16T10:10:59.830935+0000","last_fullsized":"2024-03-16T10:10:59.830935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:01:30.933303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120153+0000","last_change":"2024-03-16T10:10:58.139427+0000","last_active":"2024-03-16T10:10:59.120153+0000","last_peered":"2024-03-16T10:10:59.120153+0000","last_clean":"2024-03-16T10:10:59.120153+0000","last_became_active":"2024-03-16T10:10:58.138492+0000","last_became_peered":"2024-03-16T10:10:58.138492+0000","last_unstale":"2024-03-16T10:10:59.120153+0000","last_undegraded":"2024-03-16T10:10:59.120153+0000","last_fullsized":"2024-03-16T10:10:59.120153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116872+0000","last_change":"2024-03-16T10:10:58.141336+0000","last_active":"2024-03-16T10:10:59.116872+0000","last_peered":"2024-03-16T10:10:59.116872+0000","last_clean":"2024-03-16T10:10:59.116872+0000","last_became_active":"2024-03-16T10:10:58.139760+0000","last_became_peered":"2024-03-16T10:10:58.139760+0000","last_unstale":"2024-03-16T10:10:59.116872+0000","last_undegraded":"2024-03-16T10:10:59.116872+0000","last_fullsized":"2024-03-16T10:10:59.116872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830701+0000","last_change":"2024-03-16T10:10:58.166524+0000","last_active":"2024-03-16T10:10:59.830701+0000","last_peered":"2024-03-16T10:10:59.830701+0000","last_clean":"2024-03-16T10:10:59.830701+0000","last_became_active":"2024-03-16T10:10:58.166197+0000","last_became_peered":"2024-03-16T10:10:58.166197+0000","last_unstale":"2024-03-16T10:10:59.830701+0000","last_undegraded":"2024-03-16T10:10:59.830701+0000","last_fullsized":"2024-03-16T10:10:59.830701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466121+0000","last_change":"2024-03-16T10:10:58.138295+0000","last_active":"2024-03-16T10:10:59.466121+0000","last_peered":"2024-03-16T10:10:59.466121+0000","last_clean":"2024-03-16T10:10:59.466121+0000","last_became_active":"2024-03-16T10:10:58.136934+0000","last_became_peered":"2024-03-16T10:10:58.136934+0000","last_unstale":"2024-03-16T10:10:59.466121+0000","last_undegraded":"2024-03-16T10:10:59.466121+0000","last_fullsized":"2024-03-16T10:10:59.466121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123600+0000","last_change":"2024-03-16T10:10:58.167036+0000","last_active":"2024-03-16T10:10:59.123600+0000","last_peered":"2024-03-16T10:10:59.123600+0000","last_clean":"2024-03-16T10:10:59.123600+0000","last_became_active":"2024-03-16T10:10:58.150783+0000","last_became_peered":"2024-03-16T10:10:58.150783+0000","last_unstale":"2024-03-16T10:10:59.123600+0000","last_undegraded":"2024-03-16T10:10:59.123600+0000","last_fullsized":"2024-03-16T10:10:59.123600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:13.329157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829447+0000","last_change":"2024-03-16T10:10:58.135886+0000","last_active":"2024-03-16T10:10:59.829447+0000","last_peered":"2024-03-16T10:10:59.829447+0000","last_clean":"2024-03-16T10:10:59.829447+0000","last_became_active":"2024-03-16T10:10:58.135152+0000","last_became_peered":"2024-03-16T10:10:58.135152+0000","last_unstale":"2024-03-16T10:10:59.829447+0000","last_undegraded":"2024-03-16T10:10:59.829447+0000","last_fullsized":"2024-03-16T10:10:59.829447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:33:04.565468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.118840+0000","last_change":"2024-03-16T10:10:58.124028+0000","last_active":"2024-03-16T10:10:59.118840+0000","last_peered":"2024-03-16T10:10:59.118840+0000","last_clean":"2024-03-16T10:10:59.118840+0000","last_became_active":"2024-03-16T10:10:58.123127+0000","last_became_peered":"2024-03-16T10:10:58.123127+0000","last_unstale":"2024-03-16T10:10:59.118840+0000","last_undegraded":"2024-03-16T10:10:59.118840+0000","last_fullsized":"2024-03-16T10:10:59.118840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:18:51.093292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119320+0000","last_change":"2024-03-16T10:10:58.141873+0000","last_active":"2024-03-16T10:10:59.119320+0000","last_peered":"2024-03-16T10:10:59.119320+0000","last_clean":"2024-03-16T10:10:59.119320+0000","last_became_active":"2024-03-16T10:10:58.134577+0000","last_became_peered":"2024-03-16T10:10:58.134577+0000","last_unstale":"2024-03-16T10:10:59.119320+0000","last_undegraded":"2024-03-16T10:10:59.119320+0000","last_fullsized":"2024-03-16T10:10:59.119320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:36.702176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.831022+0000","last_change":"2024-03-16T10:10:57.116491+0000","last_active":"2024-03-16T10:10:59.831022+0000","last_peered":"2024-03-16T10:10:59.831022+0000","last_clean":"2024-03-16T10:10:59.831022+0000","last_became_active":"2024-03-16T10:10:57.116165+0000","last_became_peered":"2024-03-16T10:10:57.116165+0000","last_unstale":"2024-03-16T10:10:59.831022+0000","last_undegraded":"2024-03-16T10:10:59.831022+0000","last_fullsized":"2024-03-16T10:10:59.831022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:32:37.635494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466258+0000","last_change":"2024-03-16T10:10:58.158940+0000","last_active":"2024-03-16T10:10:59.466258+0000","last_peered":"2024-03-16T10:10:59.466258+0000","last_clean":"2024-03-16T10:10:59.466258+0000","last_became_active":"2024-03-16T10:10:58.158624+0000","last_became_peered":"2024-03-16T10:10:58.158624+0000","last_unstale":"2024-03-16T10:10:59.466258+0000","last_undegraded":"2024-03-16T10:10:59.466258+0000","last_fullsized":"2024-03-16T10:10:59.466258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120492+0000","last_change":"2024-03-16T10:10:57.103389+0000","last_active":"2024-03-16T10:10:59.120492+0000","last_peered":"2024-03-16T10:10:59.120492+0000","last_clean":"2024-03-16T10:10:59.120492+0000","last_became_active":"2024-03-16T10:10:57.100522+0000","last_became_peered":"2024-03-16T10:10:57.100522+0000","last_unstale":"2024-03-16T10:10:59.120492+0000","last_undegraded":"2024-03-16T10:10:59.120492+0000","last_fullsized":"2024-03-16T10:10:59.120492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830305+0000","last_change":"2024-03-16T10:10:57.104411+0000","last_active":"2024-03-16T10:10:59.830305+0000","last_peered":"2024-03-16T10:10:59.830305+0000","last_clean":"2024-03-16T10:10:59.830305+0000","last_became_active":"2024-03-16T10:10:57.104091+0000","last_became_peered":"2024-03-16T10:10:57.104091+0000","last_unstale":"2024-03-16T10:10:59.830305+0000","last_undegraded":"2024-03-16T10:10:59.830305+0000","last_fullsized":"2024-03-16T10:10:59.830305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:46:56.525820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829910+0000","last_change":"2024-03-16T10:10:58.133972+0000","last_active":"2024-03-16T10:10:59.829910+0000","last_peered":"2024-03-16T10:10:59.829910+0000","last_clean":"2024-03-16T10:10:59.829910+0000","last_became_active":"2024-03-16T10:10:58.131756+0000","last_became_peered":"2024-03-16T10:10:58.131756+0000","last_unstale":"2024-03-16T10:10:59.829910+0000","last_undegraded":"2024-03-16T10:10:59.829910+0000","last_fullsized":"2024-03-16T10:10:59.829910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:33:59.497661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122858+0000","last_change":"2024-03-16T10:10:57.128184+0000","last_active":"2024-03-16T10:10:59.122858+0000","last_peered":"2024-03-16T10:10:59.122858+0000","last_clean":"2024-03-16T10:10:59.122858+0000","last_became_active":"2024-03-16T10:10:57.127626+0000","last_became_peered":"2024-03-16T10:10:57.127626+0000","last_unstale":"2024-03-16T10:10:59.122858+0000","last_undegraded":"2024-03-16T10:10:59.122858+0000","last_fullsized":"2024-03-16T10:10:59.122858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.115705+0000","last_change":"2024-03-16T10:10:58.133241+0000","last_active":"2024-03-16T10:10:59.115705+0000","last_peered":"2024-03-16T10:10:59.115705+0000","last_clean":"2024-03-16T10:10:59.115705+0000","last_became_active":"2024-03-16T10:10:58.132521+0000","last_became_peered":"2024-03-16T10:10:58.132521+0000","last_unstale":"2024-03-16T10:10:59.115705+0000","last_undegraded":"2024-03-16T10:10:59.115705+0000","last_fullsized":"2024-03-16T10:10:59.115705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120090+0000","last_change":"2024-03-16T10:10:57.134408+0000","last_active":"2024-03-16T10:10:59.120090+0000","last_peered":"2024-03-16T10:10:59.120090+0000","last_clean":"2024-03-16T10:10:59.120090+0000","last_became_active":"2024-03-16T10:10:57.133909+0000","last_became_peered":"2024-03-16T10:10:57.133909+0000","last_unstale":"2024-03-16T10:10:59.120090+0000","last_undegraded":"2024-03-16T10:10:59.120090+0000","last_fullsized":"2024-03-16T10:10:59.120090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:02.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123183+0000","last_change":"2024-03-16T10:10:58.158453+0000","last_active":"2024-03-16T10:10:59.123183+0000","last_peered":"2024-03-16T10:10:59.123183+0000","last_clean":"2024-03-16T10:10:59.123183+0000","last_became_active":"2024-03-16T10:10:58.151690+0000","last_became_peered":"2024-03-16T10:10:58.151690+0000","last_unstale":"2024-03-16T10:10:59.123183+0000","last_undegraded":"2024-03-16T10:10:59.123183+0000","last_fullsized":"2024-03-16T10:10:59.123183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:08.287161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123724+0000","last_change":"2024-03-16T10:10:57.137208+0000","last_active":"2024-03-16T10:10:59.123724+0000","last_peered":"2024-03-16T10:10:59.123724+0000","last_clean":"2024-03-16T10:10:59.123724+0000","last_became_active":"2024-03-16T10:10:57.136600+0000","last_became_peered":"2024-03-16T10:10:57.136600+0000","last_unstale":"2024-03-16T10:10:59.123724+0000","last_undegraded":"2024-03-16T10:10:59.123724+0000","last_fullsized":"2024-03-16T10:10:59.123724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:02.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466338+0000","last_change":"2024-03-16T10:10:58.166872+0000","last_active":"2024-03-16T10:10:59.466338+0000","last_peered":"2024-03-16T10:10:59.466338+0000","last_clean":"2024-03-16T10:10:59.466338+0000","last_became_active":"2024-03-16T10:10:58.166556+0000","last_became_peered":"2024-03-16T10:10:58.166556+0000","last_unstale":"2024-03-16T10:10:59.466338+0000","last_undegraded":"2024-03-16T10:10:59.466338+0000","last_fullsized":"2024-03-16T10:10:59.466338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116641+0000","last_change":"2024-03-16T10:10:57.131462+0000","last_active":"2024-03-16T10:10:59.116641+0000","last_peered":"2024-03-16T10:10:59.116641+0000","last_clean":"2024-03-16T10:10:59.116641+0000","last_became_active":"2024-03-16T10:10:57.131130+0000","last_became_peered":"2024-03-16T10:10:57.131130+0000","last_unstale":"2024-03-16T10:10:59.116641+0000","last_undegraded":"2024-03-16T10:10:59.116641+0000","last_fullsized":"2024-03-16T10:10:59.116641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:02.122953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121634+0000","last_change":"2024-03-16T10:10:58.144791+0000","last_active":"2024-03-16T10:10:59.121634+0000","last_peered":"2024-03-16T10:10:59.121634+0000","last_clean":"2024-03-16T10:10:59.121634+0000","last_became_active":"2024-03-16T10:10:58.144004+0000","last_became_peered":"2024-03-16T10:10:58.144004+0000","last_unstale":"2024-03-16T10:10:59.121634+0000","last_undegraded":"2024-03-16T10:10:59.121634+0000","last_fullsized":"2024-03-16T10:10:59.121634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:54.596181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120325+0000","last_change":"2024-03-16T10:10:57.101847+0000","last_active":"2024-03-16T10:10:59.120325+0000","last_peered":"2024-03-16T10:10:59.120325+0000","last_clean":"2024-03-16T10:10:59.120325+0000","last_became_active":"2024-03-16T10:10:57.100973+0000","last_became_peered":"2024-03-16T10:10:57.100973+0000","last_unstale":"2024-03-16T10:10:59.120325+0000","last_undegraded":"2024-03-16T10:10:59.120325+0000","last_fullsized":"2024-03-16T10:10:59.120325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830301+0000","last_change":"2024-03-16T10:10:58.141884+0000","last_active":"2024-03-16T10:10:59.830301+0000","last_peered":"2024-03-16T10:10:59.830301+0000","last_clean":"2024-03-16T10:10:59.830301+0000","last_became_active":"2024-03-16T10:10:58.140254+0000","last_became_peered":"2024-03-16T10:10:58.140254+0000","last_unstale":"2024-03-16T10:10:59.830301+0000","last_undegraded":"2024-03-16T10:10:59.830301+0000","last_fullsized":"2024-03-16T10:10:59.830301+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:08.287161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466329+0000","last_change":"2024-03-16T10:10:57.117139+0000","last_active":"2024-03-16T10:10:59.466329+0000","last_peered":"2024-03-16T10:10:59.466329+0000","last_clean":"2024-03-16T10:10:59.466329+0000","last_became_active":"2024-03-16T10:10:57.111737+0000","last_became_peered":"2024-03-16T10:10:57.111737+0000","last_unstale":"2024-03-16T10:10:59.466329+0000","last_undegraded":"2024-03-16T10:10:59.466329+0000","last_fullsized":"2024-03-16T10:10:59.466329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:46:56.525820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465172+0000","last_change":"2024-03-16T10:10:58.140649+0000","last_active":"2024-03-16T10:10:59.465172+0000","last_peered":"2024-03-16T10:10:59.465172+0000","last_clean":"2024-03-16T10:10:59.465172+0000","last_became_active":"2024-03-16T10:10:58.139713+0000","last_became_peered":"2024-03-16T10:10:58.139713+0000","last_unstale":"2024-03-16T10:10:59.465172+0000","last_undegraded":"2024-03-16T10:10:59.465172+0000","last_fullsized":"2024-03-16T10:10:59.465172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:07:30.636156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123842+0000","last_change":"2024-03-16T10:10:57.138569+0000","last_active":"2024-03-16T10:10:59.123842+0000","last_peered":"2024-03-16T10:10:59.123842+0000","last_clean":"2024-03-16T10:10:59.123842+0000","last_became_active":"2024-03-16T10:10:57.136115+0000","last_became_peered":"2024-03-16T10:10:57.136115+0000","last_unstale":"2024-03-16T10:10:59.123842+0000","last_undegraded":"2024-03-16T10:10:59.123842+0000","last_fullsized":"2024-03-16T10:10:59.123842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830245+0000","last_change":"2024-03-16T10:10:58.130172+0000","last_active":"2024-03-16T10:10:59.830245+0000","last_peered":"2024-03-16T10:10:59.830245+0000","last_clean":"2024-03-16T10:10:59.830245+0000","last_became_active":"2024-03-16T10:10:58.120580+0000","last_became_peered":"2024-03-16T10:10:58.120580+0000","last_unstale":"2024-03-16T10:10:59.830245+0000","last_undegraded":"2024-03-16T10:10:59.830245+0000","last_fullsized":"2024-03-16T10:10:59.830245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:46:09.470489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120582+0000","last_change":"2024-03-16T10:10:57.104206+0000","last_active":"2024-03-16T10:10:59.120582+0000","last_peered":"2024-03-16T10:10:59.120582+0000","last_clean":"2024-03-16T10:10:59.120582+0000","last_became_active":"2024-03-16T10:10:57.100493+0000","last_became_peered":"2024-03-16T10:10:57.100493+0000","last_unstale":"2024-03-16T10:10:59.120582+0000","last_undegraded":"2024-03-16T10:10:59.120582+0000","last_fullsized":"2024-03-16T10:10:59.120582+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:32:53.097656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119275+0000","last_change":"2024-03-16T10:10:58.123895+0000","last_active":"2024-03-16T10:10:59.119275+0000","last_peered":"2024-03-16T10:10:59.119275+0000","last_clean":"2024-03-16T10:10:59.119275+0000","last_became_active":"2024-03-16T10:10:58.122933+0000","last_became_peered":"2024-03-16T10:10:58.122933+0000","last_unstale":"2024-03-16T10:10:59.119275+0000","last_undegraded":"2024-03-16T10:10:59.119275+0000","last_fullsized":"2024-03-16T10:10:59.119275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829089+0000","last_change":"2024-03-16T10:10:57.105506+0000","last_active":"2024-03-16T10:10:59.829089+0000","last_peered":"2024-03-16T10:10:59.829089+0000","last_clean":"2024-03-16T10:10:59.829089+0000","last_became_active":"2024-03-16T10:10:57.104984+0000","last_became_peered":"2024-03-16T10:10:57.104984+0000","last_unstale":"2024-03-16T10:10:59.829089+0000","last_undegraded":"2024-03-16T10:10:59.829089+0000","last_fullsized":"2024-03-16T10:10:59.829089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:34:00.287180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829799+0000","last_change":"2024-03-16T10:10:58.129371+0000","last_active":"2024-03-16T10:10:59.829799+0000","last_peered":"2024-03-16T10:10:59.829799+0000","last_clean":"2024-03-16T10:10:59.829799+0000","last_became_active":"2024-03-16T10:10:58.121147+0000","last_became_peered":"2024-03-16T10:10:58.121147+0000","last_unstale":"2024-03-16T10:10:59.829799+0000","last_undegraded":"2024-03-16T10:10:59.829799+0000","last_fullsized":"2024-03-16T10:10:59.829799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:09:11.898626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121588+0000","last_change":"2024-03-16T10:10:57.093832+0000","last_active":"2024-03-16T10:10:59.121588+0000","last_peered":"2024-03-16T10:10:59.121588+0000","last_clean":"2024-03-16T10:10:59.121588+0000","last_became_active":"2024-03-16T10:10:57.093278+0000","last_became_peered":"2024-03-16T10:10:57.093278+0000","last_unstale":"2024-03-16T10:10:59.121588+0000","last_undegraded":"2024-03-16T10:10:59.121588+0000","last_fullsized":"2024-03-16T10:10:59.121588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:35:54.941983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830807+0000","last_change":"2024-03-16T10:10:58.169395+0000","last_active":"2024-03-16T10:10:59.830807+0000","last_peered":"2024-03-16T10:10:59.830807+0000","last_clean":"2024-03-16T10:10:59.830807+0000","last_became_active":"2024-03-16T10:10:58.168702+0000","last_became_peered":"2024-03-16T10:10:58.168702+0000","last_unstale":"2024-03-16T10:10:59.830807+0000","last_undegraded":"2024-03-16T10:10:59.830807+0000","last_fullsized":"2024-03-16T10:10:59.830807+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:13.329157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829866+0000","last_change":"2024-03-16T10:10:57.105292+0000","last_active":"2024-03-16T10:10:59.829866+0000","last_peered":"2024-03-16T10:10:59.829866+0000","last_clean":"2024-03-16T10:10:59.829866+0000","last_became_active":"2024-03-16T10:10:57.104688+0000","last_became_peered":"2024-03-16T10:10:57.104688+0000","last_unstale":"2024-03-16T10:10:59.829866+0000","last_undegraded":"2024-03-16T10:10:59.829866+0000","last_fullsized":"2024-03-16T10:10:59.829866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830999+0000","last_change":"2024-03-16T10:10:58.170384+0000","last_active":"2024-03-16T10:10:59.830999+0000","last_peered":"2024-03-16T10:10:59.830999+0000","last_clean":"2024-03-16T10:10:59.830999+0000","last_became_active":"2024-03-16T10:10:58.169787+0000","last_became_peered":"2024-03-16T10:10:58.169787+0000","last_unstale":"2024-03-16T10:10:59.830999+0000","last_undegraded":"2024-03-16T10:10:59.830999+0000","last_fullsized":"2024-03-16T10:10:59.830999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:12.960572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116762+0000","last_change":"2024-03-16T10:10:57.111549+0000","last_active":"2024-03-16T10:10:59.116762+0000","last_peered":"2024-03-16T10:10:59.116762+0000","last_clean":"2024-03-16T10:10:59.116762+0000","last_became_active":"2024-03-16T10:10:57.111022+0000","last_became_peered":"2024-03-16T10:10:57.111022+0000","last_unstale":"2024-03-16T10:10:59.116762+0000","last_undegraded":"2024-03-16T10:10:59.116762+0000","last_fullsized":"2024-03-16T10:10:59.116762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:33:03.557374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123471+0000","last_change":"2024-03-16T10:10:58.150154+0000","last_active":"2024-03-16T10:10:59.123471+0000","last_peered":"2024-03-16T10:10:59.123471+0000","last_clean":"2024-03-16T10:10:59.123471+0000","last_became_active":"2024-03-16T10:10:58.149355+0000","last_became_peered":"2024-03-16T10:10:58.149355+0000","last_unstale":"2024-03-16T10:10:59.123471+0000","last_undegraded":"2024-03-16T10:10:59.123471+0000","last_fullsized":"2024-03-16T10:10:59.123471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:27:42.059927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829400+0000","last_change":"2024-03-16T10:10:57.129962+0000","last_active":"2024-03-16T10:10:59.829400+0000","last_peered":"2024-03-16T10:10:59.829400+0000","last_clean":"2024-03-16T10:10:59.829400+0000","last_became_active":"2024-03-16T10:10:57.129583+0000","last_became_peered":"2024-03-16T10:10:57.129583+0000","last_unstale":"2024-03-16T10:10:59.829400+0000","last_undegraded":"2024-03-16T10:10:59.829400+0000","last_fullsized":"2024-03-16T10:10:59.829400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:32:53.097656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830586+0000","last_change":"2024-03-16T10:10:58.140201+0000","last_active":"2024-03-16T10:10:59.830586+0000","last_peered":"2024-03-16T10:10:59.830586+0000","last_clean":"2024-03-16T10:10:59.830586+0000","last_became_active":"2024-03-16T10:10:58.139272+0000","last_became_peered":"2024-03-16T10:10:58.139272+0000","last_unstale":"2024-03-16T10:10:59.830586+0000","last_undegraded":"2024-03-16T10:10:59.830586+0000","last_fullsized":"2024-03-16T10:10:59.830586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:27:42.059927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829599+0000","last_change":"2024-03-16T10:10:48.248686+0000","last_active":"2024-03-16T10:10:59.829599+0000","last_peered":"2024-03-16T10:10:59.829599+0000","last_clean":"2024-03-16T10:10:59.829599+0000","last_became_active":"2024-03-16T10:10:48.243054+0000","last_became_peered":"2024-03-16T10:10:48.243054+0000","last_unstale":"2024-03-16T10:10:59.829599+0000","last_undegraded":"2024-03-16T10:10:59.829599+0000","last_fullsized":"2024-03-16T10:10:59.829599+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_clean_scrub_stamp":"2024-03-16T10:10:46.517938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:42.906473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.122484+0000","last_change":"2024-03-16T10:10:57.123078+0000","last_active":"2024-03-16T10:10:59.122484+0000","last_peered":"2024-03-16T10:10:59.122484+0000","last_clean":"2024-03-16T10:10:59.122484+0000","last_became_active":"2024-03-16T10:10:57.121087+0000","last_became_peered":"2024-03-16T10:10:57.121087+0000","last_unstale":"2024-03-16T10:10:59.122484+0000","last_undegraded":"2024-03-16T10:10:59.122484+0000","last_fullsized":"2024-03-16T10:10:59.122484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830505+0000","last_change":"2024-03-16T10:10:58.171123+0000","last_active":"2024-03-16T10:10:59.830505+0000","last_peered":"2024-03-16T10:10:59.830505+0000","last_clean":"2024-03-16T10:10:59.830505+0000","last_became_active":"2024-03-16T10:10:58.170849+0000","last_became_peered":"2024-03-16T10:10:58.170849+0000","last_unstale":"2024-03-16T10:10:59.830505+0000","last_undegraded":"2024-03-16T10:10:59.830505+0000","last_fullsized":"2024-03-16T10:10:59.830505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829543+0000","last_change":"2024-03-16T10:10:57.124354+0000","last_active":"2024-03-16T10:10:59.829543+0000","last_peered":"2024-03-16T10:10:59.829543+0000","last_clean":"2024-03-16T10:10:59.829543+0000","last_became_active":"2024-03-16T10:10:57.123636+0000","last_became_peered":"2024-03-16T10:10:57.123636+0000","last_unstale":"2024-03-16T10:10:59.829543+0000","last_undegraded":"2024-03-16T10:10:59.829543+0000","last_fullsized":"2024-03-16T10:10:59.829543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465205+0000","last_change":"2024-03-16T10:10:58.158168+0000","last_active":"2024-03-16T10:10:59.465205+0000","last_peered":"2024-03-16T10:10:59.465205+0000","last_clean":"2024-03-16T10:10:59.465205+0000","last_became_active":"2024-03-16T10:10:58.157882+0000","last_became_peered":"2024-03-16T10:10:58.157882+0000","last_unstale":"2024-03-16T10:10:59.465205+0000","last_undegraded":"2024-03-16T10:10:59.465205+0000","last_fullsized":"2024-03-16T10:10:59.465205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:10:11.295606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829675+0000","last_change":"2024-03-16T10:10:57.104381+0000","last_active":"2024-03-16T10:10:59.829675+0000","last_peered":"2024-03-16T10:10:59.829675+0000","last_clean":"2024-03-16T10:10:59.829675+0000","last_became_active":"2024-03-16T10:10:57.103673+0000","last_became_peered":"2024-03-16T10:10:57.103673+0000","last_unstale":"2024-03-16T10:10:59.829675+0000","last_undegraded":"2024-03-16T10:10:59.829675+0000","last_fullsized":"2024-03-16T10:10:59.829675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:45:53.158955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.115477+0000","last_change":"2024-03-16T10:10:58.136233+0000","last_active":"2024-03-16T10:10:59.115477+0000","last_peered":"2024-03-16T10:10:59.115477+0000","last_clean":"2024-03-16T10:10:59.115477+0000","last_became_active":"2024-03-16T10:10:58.135287+0000","last_became_peered":"2024-03-16T10:10:58.135287+0000","last_unstale":"2024-03-16T10:10:59.115477+0000","last_undegraded":"2024-03-16T10:10:59.115477+0000","last_fullsized":"2024-03-16T10:10:59.115477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.124008+0000","last_change":"2024-03-16T10:10:57.132978+0000","last_active":"2024-03-16T10:10:59.124008+0000","last_peered":"2024-03-16T10:10:59.124008+0000","last_clean":"2024-03-16T10:10:59.124008+0000","last_became_active":"2024-03-16T10:10:57.132199+0000","last_became_peered":"2024-03-16T10:10:57.132199+0000","last_unstale":"2024-03-16T10:10:59.124008+0000","last_undegraded":"2024-03-16T10:10:59.124008+0000","last_fullsized":"2024-03-16T10:10:59.124008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:58.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.121640+0000","last_change":"2024-03-16T10:10:58.131949+0000","last_active":"2024-03-16T10:10:59.121640+0000","last_peered":"2024-03-16T10:10:59.121640+0000","last_clean":"2024-03-16T10:10:59.121640+0000","last_became_active":"2024-03-16T10:10:58.131234+0000","last_became_peered":"2024-03-16T10:10:58.131234+0000","last_unstale":"2024-03-16T10:10:59.121640+0000","last_undegraded":"2024-03-16T10:10:59.121640+0000","last_fullsized":"2024-03-16T10:10:59.121640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:59.100283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116717+0000","last_change":"2024-03-16T10:10:57.115504+0000","last_active":"2024-03-16T10:10:59.116717+0000","last_peered":"2024-03-16T10:10:59.116717+0000","last_clean":"2024-03-16T10:10:59.116717+0000","last_became_active":"2024-03-16T10:10:57.115273+0000","last_became_peered":"2024-03-16T10:10:57.115273+0000","last_unstale":"2024-03-16T10:10:59.116717+0000","last_undegraded":"2024-03-16T10:10:59.116717+0000","last_fullsized":"2024-03-16T10:10:59.116717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:58.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123414+0000","last_change":"2024-03-16T10:10:58.150495+0000","last_active":"2024-03-16T10:10:59.123414+0000","last_peered":"2024-03-16T10:10:59.123414+0000","last_clean":"2024-03-16T10:10:59.123414+0000","last_became_active":"2024-03-16T10:10:58.149680+0000","last_became_peered":"2024-03-16T10:10:58.149680+0000","last_unstale":"2024-03-16T10:10:59.123414+0000","last_undegraded":"2024-03-16T10:10:59.123414+0000","last_fullsized":"2024-03-16T10:10:59.123414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:31.763467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830902+0000","last_change":"2024-03-16T10:10:57.128751+0000","last_active":"2024-03-16T10:10:59.830902+0000","last_peered":"2024-03-16T10:10:59.830902+0000","last_clean":"2024-03-16T10:10:59.830902+0000","last_became_active":"2024-03-16T10:10:57.127909+0000","last_became_peered":"2024-03-16T10:10:57.127909+0000","last_unstale":"2024-03-16T10:10:59.830902+0000","last_undegraded":"2024-03-16T10:10:59.830902+0000","last_fullsized":"2024-03-16T10:10:59.830902+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:09:33.897012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116753+0000","last_change":"2024-03-16T10:10:58.134860+0000","last_active":"2024-03-16T10:10:59.116753+0000","last_peered":"2024-03-16T10:10:59.116753+0000","last_clean":"2024-03-16T10:10:59.116753+0000","last_became_active":"2024-03-16T10:10:58.133470+0000","last_became_peered":"2024-03-16T10:10:58.133470+0000","last_unstale":"2024-03-16T10:10:59.116753+0000","last_undegraded":"2024-03-16T10:10:59.116753+0000","last_fullsized":"2024-03-16T10:10:59.116753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:25:52.324595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.151156+0000","last_change":"2024-03-16T10:10:57.110493+0000","last_active":"2024-03-16T10:10:59.151156+0000","last_peered":"2024-03-16T10:10:59.151156+0000","last_clean":"2024-03-16T10:10:59.151156+0000","last_became_active":"2024-03-16T10:10:57.106136+0000","last_became_peered":"2024-03-16T10:10:57.106136+0000","last_unstale":"2024-03-16T10:10:59.151156+0000","last_undegraded":"2024-03-16T10:10:59.151156+0000","last_fullsized":"2024-03-16T10:10:59.151156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.831141+0000","last_change":"2024-03-16T10:10:58.140570+0000","last_active":"2024-03-16T10:10:59.831141+0000","last_peered":"2024-03-16T10:10:59.831141+0000","last_clean":"2024-03-16T10:10:59.831141+0000","last_became_active":"2024-03-16T10:10:58.139532+0000","last_became_peered":"2024-03-16T10:10:58.139532+0000","last_unstale":"2024-03-16T10:10:59.831141+0000","last_undegraded":"2024-03-16T10:10:59.831141+0000","last_fullsized":"2024-03-16T10:10:59.831141+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:25:29.088835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120378+0000","last_change":"2024-03-16T10:10:57.120255+0000","last_active":"2024-03-16T10:10:59.120378+0000","last_peered":"2024-03-16T10:10:59.120378+0000","last_clean":"2024-03-16T10:10:59.120378+0000","last_became_active":"2024-03-16T10:10:57.119498+0000","last_became_peered":"2024-03-16T10:10:57.119498+0000","last_unstale":"2024-03-16T10:10:59.120378+0000","last_undegraded":"2024-03-16T10:10:59.120378+0000","last_fullsized":"2024-03-16T10:10:59.120378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465935+0000","last_change":"2024-03-16T10:10:58.129362+0000","last_active":"2024-03-16T10:10:59.465935+0000","last_peered":"2024-03-16T10:10:59.465935+0000","last_clean":"2024-03-16T10:10:59.465935+0000","last_became_active":"2024-03-16T10:10:58.128580+0000","last_became_peered":"2024-03-16T10:10:58.128580+0000","last_unstale":"2024-03-16T10:10:59.465935+0000","last_undegraded":"2024-03-16T10:10:59.465935+0000","last_fullsized":"2024-03-16T10:10:59.465935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:46:17.506697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.829443+0000","last_change":"2024-03-16T10:10:57.118219+0000","last_active":"2024-03-16T10:10:59.829443+0000","last_peered":"2024-03-16T10:10:59.829443+0000","last_clean":"2024-03-16T10:10:59.829443+0000","last_became_active":"2024-03-16T10:10:57.117436+0000","last_became_peered":"2024-03-16T10:10:57.117436+0000","last_unstale":"2024-03-16T10:10:59.829443+0000","last_undegraded":"2024-03-16T10:10:59.829443+0000","last_fullsized":"2024-03-16T10:10:59.829443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:07:17.676298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.119531+0000","last_change":"2024-03-16T10:10:58.143411+0000","last_active":"2024-03-16T10:10:59.119531+0000","last_peered":"2024-03-16T10:10:59.119531+0000","last_clean":"2024-03-16T10:10:59.119531+0000","last_became_active":"2024-03-16T10:10:58.142517+0000","last_became_peered":"2024-03-16T10:10:58.142517+0000","last_unstale":"2024-03-16T10:10:59.119531+0000","last_undegraded":"2024-03-16T10:10:59.119531+0000","last_fullsized":"2024-03-16T10:10:59.119531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:13:03.131047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116378+0000","last_change":"2024-03-16T10:10:57.116431+0000","last_active":"2024-03-16T10:10:59.116378+0000","last_peered":"2024-03-16T10:10:59.116378+0000","last_clean":"2024-03-16T10:10:59.116378+0000","last_became_active":"2024-03-16T10:10:57.115761+0000","last_became_peered":"2024-03-16T10:10:57.115761+0000","last_unstale":"2024-03-16T10:10:59.116378+0000","last_undegraded":"2024-03-16T10:10:59.116378+0000","last_fullsized":"2024-03-16T10:10:59.116378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:48:53.588086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830186+0000","last_change":"2024-03-16T10:10:58.137343+0000","last_active":"2024-03-16T10:10:59.830186+0000","last_peered":"2024-03-16T10:10:59.830186+0000","last_clean":"2024-03-16T10:10:59.830186+0000","last_became_active":"2024-03-16T10:10:58.136517+0000","last_became_peered":"2024-03-16T10:10:58.136517+0000","last_unstale":"2024-03-16T10:10:59.830186+0000","last_undegraded":"2024-03-16T10:10:59.830186+0000","last_fullsized":"2024-03-16T10:10:59.830186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.116455+0000","last_change":"2024-03-16T10:10:57.100812+0000","last_active":"2024-03-16T10:10:59.116455+0000","last_peered":"2024-03-16T10:10:59.116455+0000","last_clean":"2024-03-16T10:10:59.116455+0000","last_became_active":"2024-03-16T10:10:57.100312+0000","last_became_peered":"2024-03-16T10:10:57.100312+0000","last_unstale":"2024-03-16T10:10:59.116455+0000","last_undegraded":"2024-03-16T10:10:59.116455+0000","last_fullsized":"2024-03-16T10:10:59.116455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:43:35.694082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830809+0000","last_change":"2024-03-16T10:10:58.139588+0000","last_active":"2024-03-16T10:10:59.830809+0000","last_peered":"2024-03-16T10:10:59.830809+0000","last_clean":"2024-03-16T10:10:59.830809+0000","last_became_active":"2024-03-16T10:10:58.136603+0000","last_became_peered":"2024-03-16T10:10:58.136603+0000","last_unstale":"2024-03-16T10:10:59.830809+0000","last_undegraded":"2024-03-16T10:10:59.830809+0000","last_fullsized":"2024-03-16T10:10:59.830809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:44.490620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830698+0000","last_change":"2024-03-16T10:10:57.127697+0000","last_active":"2024-03-16T10:10:59.830698+0000","last_peered":"2024-03-16T10:10:59.830698+0000","last_clean":"2024-03-16T10:10:59.830698+0000","last_became_active":"2024-03-16T10:10:57.126922+0000","last_became_peered":"2024-03-16T10:10:57.126922+0000","last_unstale":"2024-03-16T10:10:59.830698+0000","last_undegraded":"2024-03-16T10:10:59.830698+0000","last_fullsized":"2024-03-16T10:10:59.830698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:56:47.775049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.120088+0000","last_change":"2024-03-16T10:10:58.162962+0000","last_active":"2024-03-16T10:10:59.120088+0000","last_peered":"2024-03-16T10:10:59.120088+0000","last_clean":"2024-03-16T10:10:59.120088+0000","last_became_active":"2024-03-16T10:10:58.161601+0000","last_became_peered":"2024-03-16T10:10:58.161601+0000","last_unstale":"2024-03-16T10:10:59.120088+0000","last_undegraded":"2024-03-16T10:10:59.120088+0000","last_fullsized":"2024-03-16T10:10:59.120088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:44:04.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123098+0000","last_change":"2024-03-16T10:10:58.160830+0000","last_active":"2024-03-16T10:10:59.123098+0000","last_peered":"2024-03-16T10:10:59.123098+0000","last_clean":"2024-03-16T10:10:59.123098+0000","last_became_active":"2024-03-16T10:10:58.160108+0000","last_became_peered":"2024-03-16T10:10:58.160108+0000","last_unstale":"2024-03-16T10:10:59.123098+0000","last_undegraded":"2024-03-16T10:10:59.123098+0000","last_fullsized":"2024-03-16T10:10:59.123098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:06:32.891269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830094+0000","last_change":"2024-03-16T10:10:57.138309+0000","last_active":"2024-03-16T10:10:59.830094+0000","last_peered":"2024-03-16T10:10:59.830094+0000","last_clean":"2024-03-16T10:10:59.830094+0000","last_became_active":"2024-03-16T10:10:57.137445+0000","last_became_peered":"2024-03-16T10:10:57.137445+0000","last_unstale":"2024-03-16T10:10:59.830094+0000","last_undegraded":"2024-03-16T10:10:59.830094+0000","last_fullsized":"2024-03-16T10:10:59.830094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:50:58.092189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.123572+0000","last_change":"2024-03-16T10:10:58.160844+0000","last_active":"2024-03-16T10:10:59.123572+0000","last_peered":"2024-03-16T10:10:59.123572+0000","last_clean":"2024-03-16T10:10:59.123572+0000","last_became_active":"2024-03-16T10:10:58.160163+0000","last_became_peered":"2024-03-16T10:10:58.160163+0000","last_unstale":"2024-03-16T10:10:59.123572+0000","last_undegraded":"2024-03-16T10:10:59.123572+0000","last_fullsized":"2024-03-16T10:10:59.123572+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:08.743197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830486+0000","last_change":"2024-03-16T10:10:57.126626+0000","last_active":"2024-03-16T10:10:59.830486+0000","last_peered":"2024-03-16T10:10:59.830486+0000","last_clean":"2024-03-16T10:10:59.830486+0000","last_became_active":"2024-03-16T10:10:57.125624+0000","last_became_peered":"2024-03-16T10:10:57.125624+0000","last_unstale":"2024-03-16T10:10:59.830486+0000","last_undegraded":"2024-03-16T10:10:59.830486+0000","last_fullsized":"2024-03-16T10:10:59.830486+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:33:03.557374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.465968+0000","last_change":"2024-03-16T10:10:58.149370+0000","last_active":"2024-03-16T10:10:59.465968+0000","last_peered":"2024-03-16T10:10:59.465968+0000","last_clean":"2024-03-16T10:10:59.465968+0000","last_became_active":"2024-03-16T10:10:58.149011+0000","last_became_peered":"2024-03-16T10:10:58.149011+0000","last_unstale":"2024-03-16T10:10:59.465968+0000","last_undegraded":"2024-03-16T10:10:59.465968+0000","last_fullsized":"2024-03-16T10:10:59.465968+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:20:35.685727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830859+0000","last_change":"2024-03-16T10:10:57.114529+0000","last_active":"2024-03-16T10:10:59.830859+0000","last_peered":"2024-03-16T10:10:59.830859+0000","last_clean":"2024-03-16T10:10:59.830859+0000","last_became_active":"2024-03-16T10:10:57.113935+0000","last_became_peered":"2024-03-16T10:10:57.113935+0000","last_unstale":"2024-03-16T10:10:59.830859+0000","last_undegraded":"2024-03-16T10:10:59.830859+0000","last_fullsized":"2024-03-16T10:10:59.830859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:34:00.287180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.830836+0000","last_change":"2024-03-16T10:10:58.138174+0000","last_active":"2024-03-16T10:10:59.830836+0000","last_peered":"2024-03-16T10:10:59.830836+0000","last_clean":"2024-03-16T10:10:59.830836+0000","last_became_active":"2024-03-16T10:10:58.136767+0000","last_became_peered":"2024-03-16T10:10:58.136767+0000","last_unstale":"2024-03-16T10:10:59.830836+0000","last_undegraded":"2024-03-16T10:10:59.830836+0000","last_fullsized":"2024-03-16T10:10:59.830836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:57.077878+0000","last_clean_scrub_stamp":"2024-03-16T10:10:57.077878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:37:40.840683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T10:10:59.466402+0000","last_change":"2024-03-16T10:10:57.124377+0000","last_active":"2024-03-16T10:10:59.466402+0000","last_peered":"2024-03-16T10:10:59.466402+0000","last_clean":"2024-03-16T10:10:59.466402+0000","last_became_active":"2024-03-16T10:10:57.123699+0000","last_became_peered":"2024-03-16T10:10:57.123699+0000","last_unstale":"2024-03-16T10:10:59.466402+0000","last_undegraded":"2024-03-16T10:10:59.466402+0000","last_fullsized":"2024-03-16T10:10:59.466402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:56.069784+0000","last_clean_scrub_stamp":"2024-03-16T10:10:56.069784+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:32:37.635494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:11:05.444 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-16T10:11:05.444 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-16T10:11:05.444 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-16T10:11:05.444 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-16T10:11:05.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.609+0000 7f7ca2bae640 1 Processor -- start 2024-03-16T10:11:05.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.609+0000 7f7ca2bae640 1 -- start start 2024-03-16T10:11:05.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.609+0000 7f7ca2bae640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c9c067220 0x7f7c9c147770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.609+0000 7f7ca2bae640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f7c9c0aff80 0x7f7c9c14eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.609+0000 7f7ca2bae640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9c0af230 con 0x7f7c9c06e080 2024-03-16T10:11:05.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.609+0000 7f7ca2bae640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f7c9c0afaf0 con 0x7f7c9c0aff80 2024-03-16T10:11:05.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.609+0000 7f7ca2bae640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f7c9c067600 con 0x7f7c9c067220 2024-03-16T10:11:05.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7c9bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7c9c06e080 0x7f7c9c13e130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56104/0 (socket says 172.21.15.17:56104) 2024-03-16T10:11:05.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7c9bfff640 1 -- 172.21.15.17:0/965797387 learned_addr learned my addr 172.21.15.17:0/965797387 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca114b640 1 --2- 172.21.15.17:0/965797387 >> v2:172.21.15.17:3300/0 conn(0x7f7c9c0aff80 0x7f7c9c14eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca094a640 1 --2- 172.21.15.17:0/965797387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c9c067220 0x7f7c9c147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca114b640 1 -- 172.21.15.17:0/965797387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c9c067220 msgr2=0x7f7c9c147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca114b640 1 --2- 172.21.15.17:0/965797387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c9c067220 0x7f7c9c147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca114b640 1 -- 172.21.15.17:0/965797387 >> v1:172.21.15.17:6789/0 conn(0x7f7c9c06e080 legacy=0x7f7c9c13e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7c9b7fe640 1 -- 172.21.15.17:0/965797387 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 889791119 0 0) 0x7f7c9c0af230 con 0x7f7c9c06e080 2024-03-16T10:11:05.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca114b640 1 -- 172.21.15.17:0/965797387 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c0b5600 con 0x7f7c9c0aff80 2024-03-16T10:11:05.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca114b640 1 --2- 172.21.15.17:0/965797387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c9c0aff80 0x7f7c9c14eb00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009d90 tx=0x7f7c9401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d4963de655dd4f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:05.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7c9b7fe640 1 -- 172.21.15.17:0/965797387 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c940b2070 con 0x7f7c9c0aff80 2024-03-16T10:11:05.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7c9b7fe640 1 -- 172.21.15.17:0/965797387 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c940a5890 con 0x7f7c9c0aff80 2024-03-16T10:11:05.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7c9b7fe640 1 -- 172.21.15.17:0/965797387 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c940a5c60 con 0x7f7c9c0aff80 2024-03-16T10:11:05.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca2bae640 1 -- 172.21.15.17:0/965797387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c9c0aff80 msgr2=0x7f7c9c14eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca2bae640 1 --2- 172.21.15.17:0/965797387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c9c0aff80 0x7f7c9c14eb00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009d90 tx=0x7f7c9401c8e0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca2bae640 1 -- 172.21.15.17:0/965797387 shutdown_connections 2024-03-16T10:11:05.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca2bae640 1 --2- 172.21.15.17:0/965797387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c9c0aff80 0x7f7c9c14eb00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca2bae640 1 --2- 172.21.15.17:0/965797387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c9c067220 0x7f7c9c147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca2bae640 1 -- 172.21.15.17:0/965797387 >> 172.21.15.17:0/965797387 conn(0x7f7c9c09b130 msgr2=0x7f7c9c09b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:05.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca2bae640 1 -- 172.21.15.17:0/965797387 shutdown_connections 2024-03-16T10:11:05.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.613+0000 7f7ca2bae640 1 -- 172.21.15.17:0/965797387 wait complete. 2024-03-16T10:11:05.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca2bae640 1 Processor -- start 2024-03-16T10:11:05.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca2bae640 1 -- start start 2024-03-16T10:11:05.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca2bae640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c9c067220 0x7f7c9c1e3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca2bae640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c9c06e080 0x7f7c9c141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca2bae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c9c0aff80 0x7f7c9c1423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca2bae640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c09bfb0 con 0x7f7c9c067220 2024-03-16T10:11:05.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca2bae640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c09be30 con 0x7f7c9c0aff80 2024-03-16T10:11:05.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca2bae640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f7c9c151860 con 0x7f7c9c06e080 2024-03-16T10:11:05.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca094a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c9c067220 0x7f7c9c1e3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca094a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c9c067220 0x7f7c9c1e3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33882/0 (socket says 172.21.15.17:33882) 2024-03-16T10:11:05.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca094a640 1 -- 172.21.15.17:0/2101974815 learned_addr learned my addr 172.21.15.17:0/2101974815 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:05.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca114b640 1 --2- 172.21.15.17:0/2101974815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c9c0aff80 0x7f7c9c1423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7c9bfff640 1 --2- 172.21.15.17:0/2101974815 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c9c06e080 0x7f7c9c141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca094a640 1 -- 172.21.15.17:0/2101974815 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c9c06e080 msgr2=0x7f7c9c141eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca094a640 1 --2- 172.21.15.17:0/2101974815 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c9c06e080 0x7f7c9c141eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca094a640 1 -- 172.21.15.17:0/2101974815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c9c0aff80 msgr2=0x7f7c9c1423f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca094a640 1 --2- 172.21.15.17:0/2101974815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c9c0aff80 0x7f7c9c1423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca094a640 1 -- 172.21.15.17:0/2101974815 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c143dd0 con 0x7f7c9c067220 2024-03-16T10:11:05.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca094a640 1 --2- 172.21.15.17:0/2101974815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c9c067220 0x7f7c9c1e3810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000d800 tx=0x7f7c9000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:05.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7c997fa640 1 -- 172.21.15.17:0/2101974815 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000b6c0 con 0x7f7c9c067220 2024-03-16T10:11:05.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7c997fa640 1 -- 172.21.15.17:0/2101974815 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c90019070 con 0x7f7c9c067220 2024-03-16T10:11:05.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca2bae640 1 -- 172.21.15.17:0/2101974815 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9c1e3fb0 con 0x7f7c9c067220 2024-03-16T10:11:05.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7c997fa640 1 -- 172.21.15.17:0/2101974815 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000ba60 con 0x7f7c9c067220 2024-03-16T10:11:05.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca2bae640 1 -- 172.21.15.17:0/2101974815 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9c14da40 con 0x7f7c9c067220 2024-03-16T10:11:05.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.617+0000 7f7ca2bae640 1 -- 172.21.15.17:0/2101974815 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9c06a130 con 0x7f7c9c067220 2024-03-16T10:11:05.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.621+0000 7f7c997fa640 1 -- 172.21.15.17:0/2101974815 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f7c9000bc00 con 0x7f7c9c067220 2024-03-16T10:11:05.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.621+0000 7f7c997fa640 1 --2- 172.21.15.17:0/2101974815 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7c88067660 0x7f7c88069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.621+0000 7f7c997fa640 1 -- 172.21.15.17:0/2101974815 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7c90086740 con 0x7f7c9c067220 2024-03-16T10:11:05.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.621+0000 7f7c9bfff640 1 --2- 172.21.15.17:0/2101974815 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7c88067660 0x7f7c88069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.621+0000 7f7c9bfff640 1 --2- 172.21.15.17:0/2101974815 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7c88067660 0x7f7c88069b20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c147920 tx=0x7f7c8c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:05.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.621+0000 7f7c997fa640 1 -- 172.21.15.17:0/2101974815 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7c90057530 con 0x7f7c9c067220 2024-03-16T10:11:05.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.813+0000 7f7ca2bae640 1 -- 172.21.15.17:0/2101974815 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7c9c0b3f30 con 0x7f7c9c067220 2024-03-16T10:11:05.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.813+0000 7f7c997fa640 1 -- 172.21.15.17:0/2101974815 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f7c90056e30 con 0x7f7c9c067220 2024-03-16T10:11:05.816 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:05.816 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-16T10:11:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.813+0000 7f7ca2bae640 1 -- 172.21.15.17:0/2101974815 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7c88067660 msgr2=0x7f7c88069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.813+0000 7f7ca2bae640 1 --2- 172.21.15.17:0/2101974815 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7c88067660 0x7f7c88069b20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c147920 tx=0x7f7c8c009040 comp rx=0 tx=0).stop 2024-03-16T10:11:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.813+0000 7f7ca2bae640 1 -- 172.21.15.17:0/2101974815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c9c067220 msgr2=0x7f7c9c1e3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.813+0000 7f7ca2bae640 1 --2- 172.21.15.17:0/2101974815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c9c067220 0x7f7c9c1e3810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000d800 tx=0x7f7c9000dcc0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.813+0000 7f7ca2bae640 1 -- 172.21.15.17:0/2101974815 shutdown_connections 2024-03-16T10:11:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.813+0000 7f7ca2bae640 1 --2- 172.21.15.17:0/2101974815 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7c88067660 0x7f7c88069b20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.813+0000 7f7ca2bae640 1 --2- 172.21.15.17:0/2101974815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c9c0aff80 0x7f7c9c1423f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.813+0000 7f7ca2bae640 1 --2- 172.21.15.17:0/2101974815 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7c9c06e080 0x7f7c9c141eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.813+0000 7f7ca2bae640 1 --2- 172.21.15.17:0/2101974815 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c9c067220 0x7f7c9c1e3810 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.813+0000 7f7ca2bae640 1 -- 172.21.15.17:0/2101974815 >> 172.21.15.17:0/2101974815 conn(0x7f7c9c09b130 msgr2=0x7f7c9c0b8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.813+0000 7f7ca2bae640 1 -- 172.21.15.17:0/2101974815 shutdown_connections 2024-03-16T10:11:05.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.813+0000 7f7ca2bae640 1 -- 172.21.15.17:0/2101974815 wait complete. 2024-03-16T10:11:05.833 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-16T10:11:05.833 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:05.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.937+0000 7ff352d73640 1 Processor -- start 2024-03-16T10:11:05.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.937+0000 7ff352d73640 1 -- start start 2024-03-16T10:11:05.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.937+0000 7ff352d73640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff34c06ee90 0x7ff34c0b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.937+0000 7ff352d73640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c06e5f0 con 0x7ff34c06e210 2024-03-16T10:11:05.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.937+0000 7ff352d73640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff34c0af160 con 0x7ff34c144400 2024-03-16T10:11:05.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.937+0000 7ff352d73640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ff34c0b0680 con 0x7ff34c06ee90 2024-03-16T10:11:05.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.937+0000 7ff350b0f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff34c06e210 0x7ff34c09ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56120/0 (socket says 172.21.15.17:56120) 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.937+0000 7ff350b0f640 1 -- 172.21.15.17:0/570468317 learned_addr learned my addr 172.21.15.17:0/570468317 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.937+0000 7ff34bfff640 1 --2- 172.21.15.17:0/570468317 >> v2:172.21.15.174:3300/0 conn(0x7ff34c06ee90 0x7ff34c0b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.937+0000 7ff34b7fe640 1 -- 172.21.15.17:0/570468317 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1586878041 0 0) 0x7ff34c06e5f0 con 0x7ff34c06e210 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.937+0000 7ff34b7fe640 1 -- 172.21.15.17:0/570468317 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff334003530 con 0x7ff34c06e210 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.937+0000 7ff34b7fe640 1 -- 172.21.15.17:0/570468317 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4050494200 0 0) 0x7ff334003530 con 0x7ff34c06e210 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 -- 172.21.15.17:0/570468317 >> v2:172.21.15.174:3300/0 conn(0x7ff34c06ee90 msgr2=0x7ff34c0b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 --2- 172.21.15.17:0/570468317 >> v2:172.21.15.174:3300/0 conn(0x7ff34c06ee90 0x7ff34c0b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 -- 172.21.15.17:0/570468317 >> v1:172.21.15.174:6789/0 conn(0x7ff34c144400 legacy=0x7ff34c144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 -- 172.21.15.17:0/570468317 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c0af440 con 0x7ff34c06e210 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 -- 172.21.15.17:0/570468317 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7ff340001680 con 0x7ff34c06e210 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 -- 172.21.15.17:0/570468317 >> v1:172.21.15.17:6789/0 conn(0x7ff34c06e210 legacy=0x7ff34c09ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 --2- 172.21.15.17:0/570468317 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3340039c0 0x7ff334003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 --2- 172.21.15.17:0/570468317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff334004810 0x7ff334004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 --2- 172.21.15.17:0/570468317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3340051d0 0x7ff33400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 -- 172.21.15.17:0/570468317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c0b0680 con 0x7ff334004810 2024-03-16T10:11:05.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 -- 172.21.15.17:0/570468317 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff33400e380 con 0x7ff3340051d0 2024-03-16T10:11:05.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 -- 172.21.15.17:0/570468317 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff33400e580 con 0x7ff3340039c0 2024-03-16T10:11:05.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34bfff640 1 --2- 172.21.15.17:0/570468317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff334004810 0x7ff334004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff351310640 1 --2- 172.21.15.17:0/570468317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3340051d0 0x7ff33400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff350b0f640 1 --2- 172.21.15.17:0/570468317 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3340039c0 0x7ff334003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34bfff640 1 -- 172.21.15.17:0/570468317 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3340039c0 msgr2=0x7ff334003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:05.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34bfff640 1 --2- 172.21.15.17:0/570468317 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3340039c0 0x7ff334003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34bfff640 1 -- 172.21.15.17:0/570468317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3340051d0 msgr2=0x7ff33400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:05.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34bfff640 1 --2- 172.21.15.17:0/570468317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3340051d0 0x7ff33400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34bfff640 1 -- 172.21.15.17:0/570468317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff334003530 con 0x7ff334004810 2024-03-16T10:11:05.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff351310640 1 -- 172.21.15.17:0/570468317 reap_dead start 2024-03-16T10:11:05.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34bfff640 1 --2- 172.21.15.17:0/570468317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff334004810 0x7ff334004c90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009ae0 tx=0x7ff33c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9e526a53f42892 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:05.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 -- 172.21.15.17:0/570468317 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0096b0 con 0x7ff334004810 2024-03-16T10:11:05.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 -- 172.21.15.17:0/570468317 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff33c0a6450 con 0x7ff334004810 2024-03-16T10:11:05.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff34b7fe640 1 -- 172.21.15.17:0/570468317 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0a6840 con 0x7ff334004810 2024-03-16T10:11:05.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff352d73640 1 -- 172.21.15.17:0/570468317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff334004810 msgr2=0x7ff334004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff352d73640 1 --2- 172.21.15.17:0/570468317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff334004810 0x7ff334004c90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff33c009ae0 tx=0x7ff33c01ca70 comp rx=0 tx=0).stop 2024-03-16T10:11:05.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff352d73640 1 -- 172.21.15.17:0/570468317 shutdown_connections 2024-03-16T10:11:05.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff352d73640 1 --2- 172.21.15.17:0/570468317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff334004810 0x7ff334004c90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff352d73640 1 -- 172.21.15.17:0/570468317 >> 172.21.15.17:0/570468317 conn(0x7ff34c0ba0e0 msgr2=0x7ff34c0ba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:05.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff352d73640 1 -- 172.21.15.17:0/570468317 shutdown_connections 2024-03-16T10:11:05.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.941+0000 7ff352d73640 1 -- 172.21.15.17:0/570468317 wait complete. 2024-03-16T10:11:05.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff352d73640 1 Processor -- start 2024-03-16T10:11:05.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff352d73640 1 -- start start 2024-03-16T10:11:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff352d73640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff334004810 0x7ff34c1dcb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff352d73640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff34c1cf6e0 0x7ff34c071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff352d73640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff34c0b10e0 0x7ff34c13ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff352d73640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c147840 con 0x7ff334004810 2024-03-16T10:11:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff352d73640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff34c1476c0 con 0x7ff34c1cf6e0 2024-03-16T10:11:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff352d73640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff34c1479c0 con 0x7ff34c0b10e0 2024-03-16T10:11:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff350b0f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff334004810 0x7ff34c1dcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff350b0f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff334004810 0x7ff34c1dcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33900/0 (socket says 172.21.15.17:33900) 2024-03-16T10:11:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff350b0f640 1 -- 172.21.15.17:0/3534702604 learned_addr learned my addr 172.21.15.17:0/3534702604 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff34bfff640 1 --2- 172.21.15.17:0/3534702604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff34c1cf6e0 0x7ff34c071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff351310640 1 --2- 172.21.15.17:0/3534702604 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff34c0b10e0 0x7ff34c13ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff350b0f640 1 -- 172.21.15.17:0/3534702604 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff34c0b10e0 msgr2=0x7ff34c13ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff350b0f640 1 --2- 172.21.15.17:0/3534702604 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff34c0b10e0 0x7ff34c13ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff350b0f640 1 -- 172.21.15.17:0/3534702604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff34c1cf6e0 msgr2=0x7ff34c071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:05.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff350b0f640 1 --2- 172.21.15.17:0/3534702604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff34c1cf6e0 0x7ff34c071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:05.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff350b0f640 1 -- 172.21.15.17:0/3534702604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff34c0b2690 con 0x7ff334004810 2024-03-16T10:11:05.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff350b0f640 1 --2- 172.21.15.17:0/3534702604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff334004810 0x7ff34c1dcb10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff34000d830 tx=0x7ff34000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:05.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff3497fa640 1 -- 172.21.15.17:0/3534702604 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff340019070 con 0x7ff334004810 2024-03-16T10:11:05.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff3497fa640 1 -- 172.21.15.17:0/3534702604 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3400074e0 con 0x7ff334004810 2024-03-16T10:11:05.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff352d73640 1 -- 172.21.15.17:0/3534702604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff34c0b15e0 con 0x7ff334004810 2024-03-16T10:11:05.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff3497fa640 1 -- 172.21.15.17:0/3534702604 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3400049f0 con 0x7ff334004810 2024-03-16T10:11:05.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff352d73640 1 -- 172.21.15.17:0/3534702604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff34c1dc310 con 0x7ff334004810 2024-03-16T10:11:05.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff352d73640 1 -- 172.21.15.17:0/3534702604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff34c06a130 con 0x7ff334004810 2024-03-16T10:11:05.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff3497fa640 1 -- 172.21.15.17:0/3534702604 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7ff34001c070 con 0x7ff334004810 2024-03-16T10:11:05.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.945+0000 7ff3497fa640 1 --2- 172.21.15.17:0/3534702604 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff338067660 0x7ff338069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:05.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.949+0000 7ff3497fa640 1 -- 172.21.15.17:0/3534702604 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff3400853b0 con 0x7ff334004810 2024-03-16T10:11:05.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.949+0000 7ff34bfff640 1 --2- 172.21.15.17:0/3534702604 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff338067660 0x7ff338069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:05.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.949+0000 7ff34bfff640 1 --2- 172.21.15.17:0/3534702604 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff338067660 0x7ff338069b20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0ad000 tx=0x7ff33c0b2040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:05.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:05.949+0000 7ff3497fa640 1 -- 172.21.15.17:0/3534702604 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff34c0b15e0 con 0x7ff334004810 2024-03-16T10:11:06.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.105+0000 7ff352d73640 1 -- 172.21.15.17:0/3534702604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff34c143400 con 0x7ff334004810 2024-03-16T10:11:06.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.105+0000 7ff3497fa640 1 -- 172.21.15.17:0/3534702604 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7ff340056180 con 0x7ff334004810 2024-03-16T10:11:06.111 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:06.111 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:06.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.109+0000 7ff352d73640 1 -- 172.21.15.17:0/3534702604 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff338067660 msgr2=0x7ff338069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.109+0000 7ff352d73640 1 --2- 172.21.15.17:0/3534702604 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff338067660 0x7ff338069b20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0ad000 tx=0x7ff33c0b2040 comp rx=0 tx=0).stop 2024-03-16T10:11:06.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.109+0000 7ff352d73640 1 -- 172.21.15.17:0/3534702604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff334004810 msgr2=0x7ff34c1dcb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.109+0000 7ff352d73640 1 --2- 172.21.15.17:0/3534702604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff334004810 0x7ff34c1dcb10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff34000d830 tx=0x7ff34000dcf0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.109+0000 7ff352d73640 1 -- 172.21.15.17:0/3534702604 shutdown_connections 2024-03-16T10:11:06.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.109+0000 7ff352d73640 1 --2- 172.21.15.17:0/3534702604 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff338067660 0x7ff338069b20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.109+0000 7ff352d73640 1 --2- 172.21.15.17:0/3534702604 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff34c0b10e0 0x7ff34c13ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.109+0000 7ff352d73640 1 --2- 172.21.15.17:0/3534702604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff34c1cf6e0 0x7ff34c071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.109+0000 7ff352d73640 1 --2- 172.21.15.17:0/3534702604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff334004810 0x7ff34c1dcb10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.109+0000 7ff352d73640 1 -- 172.21.15.17:0/3534702604 >> 172.21.15.17:0/3534702604 conn(0x7ff34c0ba0e0 msgr2=0x7ff34c0ba4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:06.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.109+0000 7ff352d73640 1 -- 172.21.15.17:0/3534702604 shutdown_connections 2024-03-16T10:11:06.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.109+0000 7ff352d73640 1 -- 172.21.15.17:0/3534702604 wait complete. 2024-03-16T10:11:06.128 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:10:55.723734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:10:56.459894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:11:06.128 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:11:06.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb20a62a640 1 Processor -- start 2024-03-16T10:11:06.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb20a62a640 1 -- start start 2024-03-16T10:11:06.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb20a62a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb204067220 0x7fb2040b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb20a62a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2041486d0 0x7fb204152040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb20a62a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2040b05a0 con 0x7fb2040b3da0 2024-03-16T10:11:06.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb20a62a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fb20406e4d0 con 0x7fb204067220 2024-03-16T10:11:06.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb20a62a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb204067600 con 0x7fb2041486d0 2024-03-16T10:11:06.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb203fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb204067220 0x7fb2040b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb2037fe640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fb2040b3da0 0x7fb204147fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.17:58720/0 (socket says 172.21.15.17:58720) 2024-03-16T10:11:06.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb2037fe640 1 -- 172.21.15.17:0/3914551188 learned_addr learned my addr 172.21.15.17:0/3914551188 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:06.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb203fff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb204067220 0x7fb2040b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.17:40306/0 (socket says 172.21.15.17:40306) 2024-03-16T10:11:06.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb208bc7640 1 --2- 172.21.15.17:0/3914551188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2041486d0 0x7fb204152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb203fff640 1 -- 172.21.15.17:0/3914551188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2041486d0 msgr2=0x7fb204152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:06.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb203fff640 1 --2- 172.21.15.17:0/3914551188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2041486d0 0x7fb204152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb203fff640 1 -- 172.21.15.17:0/3914551188 >> v1:172.21.15.174:6789/0 conn(0x7fb2040b3da0 legacy=0x7fb204147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb203fff640 1 -- 172.21.15.17:0/3914551188 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20406e1f0 con 0x7fb204067220 2024-03-16T10:11:06.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.249+0000 7fb208bc7640 1 --2- 172.21.15.17:0/3914551188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2041486d0 0x7fb204152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:11:06.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.253+0000 7fb203fff640 1 --2- 172.21.15.17:0/3914551188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb204067220 0x7fb2040b0060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40097c0 tx=0x7fb1f401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ba29a5c31ffdc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:06.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.253+0000 7fb202ffd640 1 -- 172.21.15.17:0/3914551188 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f40b0070 con 0x7fb204067220 2024-03-16T10:11:06.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.253+0000 7fb202ffd640 1 -- 172.21.15.17:0/3914551188 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f40ab600 con 0x7fb204067220 2024-03-16T10:11:06.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.253+0000 7fb202ffd640 1 -- 172.21.15.17:0/3914551188 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f40ab9d0 con 0x7fb204067220 2024-03-16T10:11:06.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.253+0000 7fb20a62a640 1 -- 172.21.15.17:0/3914551188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb204067220 msgr2=0x7fb2040b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.253+0000 7fb20a62a640 1 --2- 172.21.15.17:0/3914551188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb204067220 0x7fb2040b0060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40097c0 tx=0x7fb1f401c4a0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.253+0000 7fb20a62a640 1 -- 172.21.15.17:0/3914551188 shutdown_connections 2024-03-16T10:11:06.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.253+0000 7fb20a62a640 1 --2- 172.21.15.17:0/3914551188 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2041486d0 0x7fb204152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.253+0000 7fb20a62a640 1 --2- 172.21.15.17:0/3914551188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb204067220 0x7fb2040b0060 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.253+0000 7fb20a62a640 1 -- 172.21.15.17:0/3914551188 >> 172.21.15.17:0/3914551188 conn(0x7fb2040b5ea0 msgr2=0x7fb2040b6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:06.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.253+0000 7fb20a62a640 1 -- 172.21.15.17:0/3914551188 shutdown_connections 2024-03-16T10:11:06.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.253+0000 7fb20a62a640 1 -- 172.21.15.17:0/3914551188 wait complete. 2024-03-16T10:11:06.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb20a62a640 1 Processor -- start 2024-03-16T10:11:06.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb20a62a640 1 -- start start 2024-03-16T10:11:06.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb20a62a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb204067220 0x7fb204144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb20a62a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2040b3da0 0x7fb2041453d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb20a62a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2041486d0 0x7fb2041e39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb20a62a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb204154da0 con 0x7fb204067220 2024-03-16T10:11:06.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb20a62a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb2040b4590 con 0x7fb2041486d0 2024-03-16T10:11:06.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb20a62a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb204154f20 con 0x7fb2040b3da0 2024-03-16T10:11:06.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb203fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb204067220 0x7fb204144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb203fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb204067220 0x7fb204144e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33902/0 (socket says 172.21.15.17:33902) 2024-03-16T10:11:06.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb203fff640 1 -- 172.21.15.17:0/3367264957 learned_addr learned my addr 172.21.15.17:0/3367264957 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:06.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb208bc7640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2041486d0 0x7fb2041e39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb2037fe640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2040b3da0 0x7fb2041453d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb203fff640 1 -- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2040b3da0 msgr2=0x7fb2041453d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb203fff640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2040b3da0 0x7fb2041453d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb203fff640 1 -- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2041486d0 msgr2=0x7fb2041e39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb203fff640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2041486d0 0x7fb2041e39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb203fff640 1 -- 172.21.15.17:0/3367264957 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb204071440 con 0x7fb204067220 2024-03-16T10:11:06.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb2037fe640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2040b3da0 0x7fb2041453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:06.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb203fff640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb204067220 0x7fb204144e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40ae040 tx=0x7fb1f401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:06.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb200ff9640 1 -- 172.21.15.17:0/3367264957 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f40b0050 con 0x7fb204067220 2024-03-16T10:11:06.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb200ff9640 1 -- 172.21.15.17:0/3367264957 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f40bc070 con 0x7fb204067220 2024-03-16T10:11:06.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb200ff9640 1 -- 172.21.15.17:0/3367264957 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f40b5830 con 0x7fb204067220 2024-03-16T10:11:06.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb20a62a640 1 -- 172.21.15.17:0/3367264957 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb204145910 con 0x7fb204067220 2024-03-16T10:11:06.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb20a62a640 1 -- 172.21.15.17:0/3367264957 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2040727d0 con 0x7fb204067220 2024-03-16T10:11:06.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.257+0000 7fb20a62a640 1 -- 172.21.15.17:0/3367264957 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20406a130 con 0x7fb204067220 2024-03-16T10:11:06.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.261+0000 7fb200ff9640 1 -- 172.21.15.17:0/3367264957 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fb1f40b59d0 con 0x7fb204067220 2024-03-16T10:11:06.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.261+0000 7fb200ff9640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb1f0067660 0x7fb1f0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.261+0000 7fb200ff9640 1 -- 172.21.15.17:0/3367264957 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb1f411ff70 con 0x7fb204067220 2024-03-16T10:11:06.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.261+0000 7fb2037fe640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb1f0067660 0x7fb1f0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.261+0000 7fb2037fe640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb1f0067660 0x7fb1f0069b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8007fd0 tx=0x7fb1f8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:06.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.269+0000 7fb200ff9640 1 -- 172.21.15.17:0/3367264957 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb1f4016090 con 0x7fb204067220 2024-03-16T10:11:06.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.441+0000 7fb20a62a640 1 -- 172.21.15.17:0/3367264957 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb204099da0 con 0x7fb204067220 2024-03-16T10:11:06.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.445+0000 7fb200ff9640 1 -- 172.21.15.17:0/3367264957 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fb1f40f0da0 con 0x7fb204067220 2024-03-16T10:11:06.447 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:11:06.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.445+0000 7fb20a62a640 1 -- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb1f0067660 msgr2=0x7fb1f0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.445+0000 7fb20a62a640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb1f0067660 0x7fb1f0069b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8007fd0 tx=0x7fb1f8009040 comp rx=0 tx=0).stop 2024-03-16T10:11:06.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.445+0000 7fb20a62a640 1 -- 172.21.15.17:0/3367264957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb204067220 msgr2=0x7fb204144e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.445+0000 7fb20a62a640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb204067220 0x7fb204144e90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb1f40ae040 tx=0x7fb1f401ace0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.445+0000 7fb20a62a640 1 -- 172.21.15.17:0/3367264957 shutdown_connections 2024-03-16T10:11:06.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.445+0000 7fb20a62a640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb1f0067660 0x7fb1f0069b20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.445+0000 7fb20a62a640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2041486d0 0x7fb2041e39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.445+0000 7fb20a62a640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2040b3da0 0x7fb2041453d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.445+0000 7fb20a62a640 1 --2- 172.21.15.17:0/3367264957 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb204067220 0x7fb204144e90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.445+0000 7fb20a62a640 1 -- 172.21.15.17:0/3367264957 >> 172.21.15.17:0/3367264957 conn(0x7fb2040b5ea0 msgr2=0x7fb20409ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:06.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.445+0000 7fb20a62a640 1 -- 172.21.15.17:0/3367264957 shutdown_connections 2024-03-16T10:11:06.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.445+0000 7fb20a62a640 1 -- 172.21.15.17:0/3367264957 wait complete. 2024-03-16T10:11:06.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:11:06.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f32a7d640 1 Processor -- start 2024-03-16T10:11:06.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f32a7d640 1 -- start start 2024-03-16T10:11:06.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f32a7d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2f2c0b49d0 0x7f2f2c150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f32a7d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f2c0b6fd0 con 0x7f2f2c073070 2024-03-16T10:11:06.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f32a7d640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f2c0b64f0 con 0x7f2f2c0b7770 2024-03-16T10:11:06.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f32a7d640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f2f2c0b67d0 con 0x7f2f2c0b49d0 2024-03-16T10:11:06.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2f2c073070 0x7f2f2c0b42c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56128/0 (socket says 172.21.15.17:56128) 2024-03-16T10:11:06.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2bfff640 1 -- 172.21.15.17:0/3078294810 learned_addr learned my addr 172.21.15.17:0/3078294810 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:06.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f3101a640 1 --2- 172.21.15.17:0/3078294810 >> v2:172.21.15.174:3300/0 conn(0x7f2f2c0b49d0 0x7f2f2c150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 -- 172.21.15.17:0/3078294810 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 461803768 0 0) 0x7f2f2c0b6fd0 con 0x7f2f2c073070 2024-03-16T10:11:06.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 -- 172.21.15.17:0/3078294810 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f1c003530 con 0x7f2f2c073070 2024-03-16T10:11:06.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 -- 172.21.15.17:0/3078294810 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3224850041 0 0) 0x7f2f1c003530 con 0x7f2f2c073070 2024-03-16T10:11:06.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 -- 172.21.15.17:0/3078294810 >> v2:172.21.15.174:3300/0 conn(0x7f2f2c0b49d0 msgr2=0x7f2f2c150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:06.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 --2- 172.21.15.17:0/3078294810 >> v2:172.21.15.174:3300/0 conn(0x7f2f2c0b49d0 0x7f2f2c150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 -- 172.21.15.17:0/3078294810 >> v1:172.21.15.174:6789/0 conn(0x7f2f2c0b7770 legacy=0x7f2f2c145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 -- 172.21.15.17:0/3078294810 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2c0b6ab0 con 0x7f2f2c073070 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 -- 172.21.15.17:0/3078294810 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f2f18001680 con 0x7f2f2c073070 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 -- 172.21.15.17:0/3078294810 >> v1:172.21.15.17:6789/0 conn(0x7f2f2c073070 legacy=0x7f2f2c0b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 --2- 172.21.15.17:0/3078294810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f1c003a10 0x7f2f1c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 --2- 172.21.15.17:0/3078294810 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f1c004810 0x7f2f1c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 --2- 172.21.15.17:0/3078294810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f1c0053d0 0x7f2f1c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 -- 172.21.15.17:0/3078294810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c0b67d0 con 0x7f2f1c003a10 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 -- 172.21.15.17:0/3078294810 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c00c2b0 con 0x7f2f1c0053d0 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2b7fe640 1 -- 172.21.15.17:0/3078294810 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2f1c00c4b0 con 0x7f2f1c004810 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f30819640 1 --2- 172.21.15.17:0/3078294810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f1c003a10 0x7f2f1c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f2bfff640 1 --2- 172.21.15.17:0/3078294810 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f1c004810 0x7f2f1c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f30819640 1 -- 172.21.15.17:0/3078294810 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f1c004810 msgr2=0x7f2f1c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f3101a640 1 --2- 172.21.15.17:0/3078294810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f1c0053d0 0x7f2f1c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f30819640 1 --2- 172.21.15.17:0/3078294810 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f1c004810 0x7f2f1c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f30819640 1 -- 172.21.15.17:0/3078294810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f1c0053d0 msgr2=0x7f2f1c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f30819640 1 --2- 172.21.15.17:0/3078294810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f1c0053d0 0x7f2f1c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f30819640 1 -- 172.21.15.17:0/3078294810 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1c003530 con 0x7f2f1c003a10 2024-03-16T10:11:06.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.589+0000 7f2f3101a640 1 -- 172.21.15.17:0/3078294810 reap_dead start 2024-03-16T10:11:06.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f30819640 1 --2- 172.21.15.17:0/3078294810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f1c003a10 0x7f2f1c003e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2f20001780 tx=0x7f2f2001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a60603b711ba700 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:06.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f2b7fe640 1 -- 172.21.15.17:0/3078294810 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f200013a0 con 0x7f2f1c003a10 2024-03-16T10:11:06.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f2b7fe640 1 -- 172.21.15.17:0/3078294810 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f200052f0 con 0x7f2f1c003a10 2024-03-16T10:11:06.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f2b7fe640 1 -- 172.21.15.17:0/3078294810 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20003840 con 0x7f2f1c003a10 2024-03-16T10:11:06.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 -- 172.21.15.17:0/3078294810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f1c003a10 msgr2=0x7f2f1c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 --2- 172.21.15.17:0/3078294810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f1c003a10 0x7f2f1c003e30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2f20001780 tx=0x7f2f2001c400 comp rx=0 tx=0).stop 2024-03-16T10:11:06.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 -- 172.21.15.17:0/3078294810 shutdown_connections 2024-03-16T10:11:06.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 --2- 172.21.15.17:0/3078294810 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f1c003a10 0x7f2f1c003e30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 -- 172.21.15.17:0/3078294810 >> 172.21.15.17:0/3078294810 conn(0x7f2f2c06b290 msgr2=0x7f2f2c06b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:06.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 -- 172.21.15.17:0/3078294810 shutdown_connections 2024-03-16T10:11:06.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 -- 172.21.15.17:0/3078294810 wait complete. 2024-03-16T10:11:06.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 Processor -- start 2024-03-16T10:11:06.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 -- start start 2024-03-16T10:11:06.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f1c003a10 0x7f2f2c06dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f2c146d00 0x7f2f2c147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f2c13c330 0x7f2f2c13c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c148770 con 0x7f2f1c003a10 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2c145a10 con 0x7f2f2c146d00 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f32a7d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2f2c1488f0 con 0x7f2f2c13c330 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f30819640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f1c003a10 0x7f2f2c06dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f30819640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f1c003a10 0x7f2f2c06dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33920/0 (socket says 172.21.15.17:33920) 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f30819640 1 -- 172.21.15.17:0/870111024 learned_addr learned my addr 172.21.15.17:0/870111024 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f2bfff640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f2c146d00 0x7f2f2c147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f3101a640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f2c13c330 0x7f2f2c13c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f30819640 1 -- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f2c13c330 msgr2=0x7f2f2c13c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f30819640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f2c13c330 0x7f2f2c13c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f30819640 1 -- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f2c146d00 msgr2=0x7f2f2c147180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f30819640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f2c146d00 0x7f2f2c147180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f30819640 1 -- 172.21.15.17:0/870111024 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2c06d6c0 con 0x7f2f1c003a10 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.593+0000 7f2f2bfff640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f2c146d00 0x7f2f2c147180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.597+0000 7f2f30819640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f1c003a10 0x7f2f2c06dbb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2f20005040 tx=0x7f2f1c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:06.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.597+0000 7f2f297fa640 1 -- 172.21.15.17:0/870111024 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f200b8020 con 0x7f2f1c003a10 2024-03-16T10:11:06.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.597+0000 7f2f297fa640 1 -- 172.21.15.17:0/870111024 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f200054a0 con 0x7f2f1c003a10 2024-03-16T10:11:06.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.597+0000 7f2f32a7d640 1 -- 172.21.15.17:0/870111024 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2c1476c0 con 0x7f2f1c003a10 2024-03-16T10:11:06.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.597+0000 7f2f297fa640 1 -- 172.21.15.17:0/870111024 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f200013a0 con 0x7f2f1c003a10 2024-03-16T10:11:06.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.597+0000 7f2f32a7d640 1 -- 172.21.15.17:0/870111024 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f2c13ab20 con 0x7f2f1c003a10 2024-03-16T10:11:06.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.597+0000 7f2f32a7d640 1 -- 172.21.15.17:0/870111024 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f2c06a130 con 0x7f2f1c003a10 2024-03-16T10:11:06.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.597+0000 7f2f297fa640 1 -- 172.21.15.17:0/870111024 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f2f200bb020 con 0x7f2f1c003a10 2024-03-16T10:11:06.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.597+0000 7f2f297fa640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2f10067690 0x7f2f10069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.597+0000 7f2f297fa640 1 -- 172.21.15.17:0/870111024 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2f2011f6d0 con 0x7f2f1c003a10 2024-03-16T10:11:06.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.597+0000 7f2f2bfff640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2f10067690 0x7f2f10069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.597+0000 7f2f2bfff640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2f10067690 0x7f2f10069b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2f18000fc0 tx=0x7f2f18009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:06.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.601+0000 7f2f297fa640 1 -- 172.21.15.17:0/870111024 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2f200c2030 con 0x7f2f1c003a10 2024-03-16T10:11:06.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.761+0000 7f2f32a7d640 1 -- 172.21.15.17:0/870111024 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2f2c0b7e10 con 0x7f2f1c003a10 2024-03-16T10:11:06.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.761+0000 7f2f297fa640 1 -- 172.21.15.17:0/870111024 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f2f2c1476c0 con 0x7f2f1c003a10 2024-03-16T10:11:06.765 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-16T10:11:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.765+0000 7f2f32a7d640 1 -- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2f10067690 msgr2=0x7f2f10069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.765+0000 7f2f32a7d640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2f10067690 0x7f2f10069b50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2f18000fc0 tx=0x7f2f18009040 comp rx=0 tx=0).stop 2024-03-16T10:11:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.765+0000 7f2f32a7d640 1 -- 172.21.15.17:0/870111024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f1c003a10 msgr2=0x7f2f2c06dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.765+0000 7f2f32a7d640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f1c003a10 0x7f2f2c06dbb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2f20005040 tx=0x7f2f1c005840 comp rx=0 tx=0).stop 2024-03-16T10:11:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.765+0000 7f2f32a7d640 1 -- 172.21.15.17:0/870111024 shutdown_connections 2024-03-16T10:11:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.765+0000 7f2f32a7d640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2f10067690 0x7f2f10069b50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.765+0000 7f2f32a7d640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2f2c13c330 0x7f2f2c13c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.765+0000 7f2f32a7d640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f2c146d00 0x7f2f2c147180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.765+0000 7f2f32a7d640 1 --2- 172.21.15.17:0/870111024 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f1c003a10 0x7f2f2c06dbb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.765+0000 7f2f32a7d640 1 -- 172.21.15.17:0/870111024 >> 172.21.15.17:0/870111024 conn(0x7f2f2c06b290 msgr2=0x7f2f2c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.765+0000 7f2f32a7d640 1 -- 172.21.15.17:0/870111024 shutdown_connections 2024-03-16T10:11:06.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.765+0000 7f2f32a7d640 1 -- 172.21.15.17:0/870111024 wait complete. 2024-03-16T10:11:06.784 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:11:06.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec231e3640 1 Processor -- start 2024-03-16T10:11:06.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec231e3640 1 -- start start 2024-03-16T10:11:06.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec231e3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fec1c0b5740 0x7fec1c0aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec231e3640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fec1c14c9a0 0x7fec1c14ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec231e3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec1c0b5b20 con 0x7fec1c06ede0 2024-03-16T10:11:06.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec231e3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fec1c06e950 con 0x7fec1c0b5740 2024-03-16T10:11:06.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec231e3640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fec1c0af720 con 0x7fec1c14c9a0 2024-03-16T10:11:06.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec1bfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fec1c06ede0 0x7fec1c071ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56142/0 (socket says 172.21.15.17:56142) 2024-03-16T10:11:06.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec1bfff640 1 -- 172.21.15.17:0/278151786 learned_addr learned my addr 172.21.15.17:0/278151786 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:06.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec20f7f640 1 --2- 172.21.15.17:0/278151786 >> v2:172.21.15.17:3300/0 conn(0x7fec1c0b5740 0x7fec1c0aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec21780640 1 --2- 172.21.15.17:0/278151786 >> v2:172.21.15.174:3300/0 conn(0x7fec1c14c9a0 0x7fec1c14ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec1b7fe640 1 -- 172.21.15.17:0/278151786 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2252321494 0 0) 0x7fec1c0b5b20 con 0x7fec1c06ede0 2024-03-16T10:11:06.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec20f7f640 1 -- 172.21.15.17:0/278151786 >> v2:172.21.15.174:3300/0 conn(0x7fec1c14c9a0 msgr2=0x7fec1c14ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:06.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec20f7f640 1 --2- 172.21.15.17:0/278151786 >> v2:172.21.15.174:3300/0 conn(0x7fec1c14c9a0 0x7fec1c14ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec20f7f640 1 -- 172.21.15.17:0/278151786 >> v1:172.21.15.17:6789/0 conn(0x7fec1c06ede0 legacy=0x7fec1c071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec20f7f640 1 -- 172.21.15.17:0/278151786 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c0afa00 con 0x7fec1c0b5740 2024-03-16T10:11:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.901+0000 7fec20f7f640 1 --2- 172.21.15.17:0/278151786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec1c0b5740 0x7fec1c0aeaf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0706a0 tx=0x7fec1001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e65ebc2512df83 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec1b7fe640 1 -- 172.21.15.17:0/278151786 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec100b1070 con 0x7fec1c0b5740 2024-03-16T10:11:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec1b7fe640 1 -- 172.21.15.17:0/278151786 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec100ac5f0 con 0x7fec1c0b5740 2024-03-16T10:11:06.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec1b7fe640 1 -- 172.21.15.17:0/278151786 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec100ac9c0 con 0x7fec1c0b5740 2024-03-16T10:11:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 -- 172.21.15.17:0/278151786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec1c0b5740 msgr2=0x7fec1c0aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 --2- 172.21.15.17:0/278151786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec1c0b5740 0x7fec1c0aeaf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0706a0 tx=0x7fec1001c480 comp rx=0 tx=0).stop 2024-03-16T10:11:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 -- 172.21.15.17:0/278151786 shutdown_connections 2024-03-16T10:11:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 --2- 172.21.15.17:0/278151786 >> v2:172.21.15.174:3300/0 conn(0x7fec1c14c9a0 0x7fec1c14ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 --2- 172.21.15.17:0/278151786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec1c0b5740 0x7fec1c0aeaf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 -- 172.21.15.17:0/278151786 >> 172.21.15.17:0/278151786 conn(0x7fec1c0b65d0 msgr2=0x7fec1c0b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 -- 172.21.15.17:0/278151786 shutdown_connections 2024-03-16T10:11:06.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 -- 172.21.15.17:0/278151786 wait complete. 2024-03-16T10:11:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 Processor -- start 2024-03-16T10:11:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 -- start start 2024-03-16T10:11:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec1c06ede0 0x7fec1c13f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec1c0b5740 0x7fec1c1de160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec1c14c9a0 0x7fec1c1de6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c1477a0 con 0x7fec1c06ede0 2024-03-16T10:11:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c147620 con 0x7fec1c0b5740 2024-03-16T10:11:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fec1c0725e0 con 0x7fec1c14c9a0 2024-03-16T10:11:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec20f7f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec1c06ede0 0x7fec1c13f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec20f7f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec1c06ede0 0x7fec1c13f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33930/0 (socket says 172.21.15.17:33930) 2024-03-16T10:11:06.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec20f7f640 1 -- 172.21.15.17:0/2317577318 learned_addr learned my addr 172.21.15.17:0/2317577318 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec1bfff640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec1c0b5740 0x7fec1c1de160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec21780640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec1c14c9a0 0x7fec1c1de6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec20f7f640 1 -- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec1c14c9a0 msgr2=0x7fec1c1de6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec20f7f640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec1c14c9a0 0x7fec1c1de6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec20f7f640 1 -- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec1c0b5740 msgr2=0x7fec1c1de160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec20f7f640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec1c0b5740 0x7fec1c1de160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec20f7f640 1 -- 172.21.15.17:0/2317577318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c0b0fe0 con 0x7fec1c06ede0 2024-03-16T10:11:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec21780640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec1c14c9a0 0x7fec1c1de6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec1bfff640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec1c0b5740 0x7fec1c1de160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec20f7f640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec1c06ede0 0x7fec1c13f680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fec1001ab30 tx=0x7fec10019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec197fa640 1 -- 172.21.15.17:0/2317577318 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec100b9020 con 0x7fec1c06ede0 2024-03-16T10:11:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec197fa640 1 -- 172.21.15.17:0/2317577318 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec100b5410 con 0x7fec1c06ede0 2024-03-16T10:11:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 -- 172.21.15.17:0/2317577318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1c140d40 con 0x7fec1c06ede0 2024-03-16T10:11:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec197fa640 1 -- 172.21.15.17:0/2317577318 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec100b1040 con 0x7fec1c06ede0 2024-03-16T10:11:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.905+0000 7fec231e3640 1 -- 172.21.15.17:0/2317577318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1c0b1250 con 0x7fec1c06ede0 2024-03-16T10:11:06.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.909+0000 7fec231e3640 1 -- 172.21.15.17:0/2317577318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1c1dde30 con 0x7fec1c06ede0 2024-03-16T10:11:06.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.909+0000 7fec197fa640 1 -- 172.21.15.17:0/2317577318 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fec100bc020 con 0x7fec1c06ede0 2024-03-16T10:11:06.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.909+0000 7fec197fa640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fec00067660 0x7fec00069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:06.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.909+0000 7fec197fa640 1 -- 172.21.15.17:0/2317577318 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fec1011f720 con 0x7fec1c06ede0 2024-03-16T10:11:06.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.909+0000 7fec1bfff640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fec00067660 0x7fec00069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:06.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.909+0000 7fec1bfff640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fec00067660 0x7fec00069b20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec08007fd0 tx=0x7fec08009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:06.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:06.913+0000 7fec197fa640 1 -- 172.21.15.17:0/2317577318 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fec100c3030 con 0x7fec1c06ede0 2024-03-16T10:11:07.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.065+0000 7fec231e3640 1 -- 172.21.15.17:0/2317577318 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fec1c06a130 con 0x7fec1c06ede0 2024-03-16T10:11:07.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.069+0000 7fec197fa640 1 -- 172.21.15.17:0/2317577318 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fec100a9070 con 0x7fec1c06ede0 2024-03-16T10:11:07.071 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-16T10:11:07.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.069+0000 7fec231e3640 1 -- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fec00067660 msgr2=0x7fec00069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.069+0000 7fec231e3640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fec00067660 0x7fec00069b20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fec08007fd0 tx=0x7fec08009040 comp rx=0 tx=0).stop 2024-03-16T10:11:07.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.069+0000 7fec231e3640 1 -- 172.21.15.17:0/2317577318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec1c06ede0 msgr2=0x7fec1c13f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.069+0000 7fec231e3640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec1c06ede0 0x7fec1c13f680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fec1001ab30 tx=0x7fec10019220 comp rx=0 tx=0).stop 2024-03-16T10:11:07.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.069+0000 7fec231e3640 1 -- 172.21.15.17:0/2317577318 shutdown_connections 2024-03-16T10:11:07.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.069+0000 7fec231e3640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fec00067660 0x7fec00069b20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.069+0000 7fec231e3640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec1c14c9a0 0x7fec1c1de6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.069+0000 7fec231e3640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec1c0b5740 0x7fec1c1de160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.069+0000 7fec231e3640 1 --2- 172.21.15.17:0/2317577318 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec1c06ede0 0x7fec1c13f680 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.069+0000 7fec231e3640 1 -- 172.21.15.17:0/2317577318 >> 172.21.15.17:0/2317577318 conn(0x7fec1c0b65d0 msgr2=0x7fec1c06dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:07.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.069+0000 7fec231e3640 1 -- 172.21.15.17:0/2317577318 shutdown_connections 2024-03-16T10:11:07.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.069+0000 7fec231e3640 1 -- 172.21.15.17:0/2317577318 wait complete. 2024-03-16T10:11:07.090 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:11:07.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.213+0000 7fd395686640 1 Processor -- start 2024-03-16T10:11:07.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.213+0000 7fd395686640 1 -- start start 2024-03-16T10:11:07.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd395686640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd390067220 0x7fd3901478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd395686640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd3900b4e50 0x7fd390147e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd395686640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd3901414d0 0x7fd390141920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd395686640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd3900af150 con 0x7fd390067220 2024-03-16T10:11:07.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd395686640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fd3900aeee0 con 0x7fd3900b4e50 2024-03-16T10:11:07.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd395686640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd390067600 con 0x7fd3901414d0 2024-03-16T10:11:07.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd38effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd390067220 0x7fd3901478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd38effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd390067220 0x7fd3901478d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33946/0 (socket says 172.21.15.17:33946) 2024-03-16T10:11:07.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd38effd640 1 -- 172.21.15.17:0/2694401553 learned_addr learned my addr 172.21.15.17:0/2694401553 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:07.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd38e7fc640 1 --2- 172.21.15.17:0/2694401553 >> v2:172.21.15.174:3300/0 conn(0x7fd3900b4e50 0x7fd390147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd38f7fe640 1 --2- 172.21.15.17:0/2694401553 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd3901414d0 0x7fd390141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd38effd640 1 -- 172.21.15.17:0/2694401553 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd3901414d0 msgr2=0x7fd390141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:07.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd38effd640 1 --2- 172.21.15.17:0/2694401553 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd3901414d0 0x7fd390141920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd38effd640 1 -- 172.21.15.17:0/2694401553 >> v2:172.21.15.174:3300/0 conn(0x7fd3900b4e50 msgr2=0x7fd390147e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:07.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd38effd640 1 --2- 172.21.15.17:0/2694401553 >> v2:172.21.15.174:3300/0 conn(0x7fd3900b4e50 0x7fd390147e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd38effd640 1 -- 172.21.15.17:0/2694401553 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd390141e60 con 0x7fd390067220 2024-03-16T10:11:07.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd38effd640 1 --2- 172.21.15.17:0/2694401553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390067220 0x7fd3901478d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd3900ae460 tx=0x7fd38801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5071f6669209063e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:07.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd38dffb640 1 -- 172.21.15.17:0/2694401553 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3880ab650 con 0x7fd390067220 2024-03-16T10:11:07.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd38dffb640 1 -- 172.21.15.17:0/2694401553 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3880b0070 con 0x7fd390067220 2024-03-16T10:11:07.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd38dffb640 1 -- 172.21.15.17:0/2694401553 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3880aba10 con 0x7fd390067220 2024-03-16T10:11:07.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd395686640 1 -- 172.21.15.17:0/2694401553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390067220 msgr2=0x7fd3901478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd395686640 1 --2- 172.21.15.17:0/2694401553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390067220 0x7fd3901478d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd3900ae460 tx=0x7fd38801c490 comp rx=0 tx=0).stop 2024-03-16T10:11:07.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd395686640 1 -- 172.21.15.17:0/2694401553 shutdown_connections 2024-03-16T10:11:07.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd395686640 1 --2- 172.21.15.17:0/2694401553 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd3901414d0 0x7fd390141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd395686640 1 --2- 172.21.15.17:0/2694401553 >> v2:172.21.15.174:3300/0 conn(0x7fd3900b4e50 0x7fd390147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd395686640 1 --2- 172.21.15.17:0/2694401553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd390067220 0x7fd3901478d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.217+0000 7fd395686640 1 -- 172.21.15.17:0/2694401553 >> 172.21.15.17:0/2694401553 conn(0x7fd39006bd70 msgr2=0x7fd3900b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:07.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd395686640 1 -- 172.21.15.17:0/2694401553 shutdown_connections 2024-03-16T10:11:07.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd395686640 1 -- 172.21.15.17:0/2694401553 wait complete. 2024-03-16T10:11:07.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd395686640 1 Processor -- start 2024-03-16T10:11:07.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd395686640 1 -- start start 2024-03-16T10:11:07.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd395686640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd390067220 0x7fd39013e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd395686640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3900b4e50 0x7fd39013cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd395686640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3901414d0 0x7fd390138340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd395686640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd39009e890 con 0x7fd3900b4e50 2024-03-16T10:11:07.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd395686640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd39009e710 con 0x7fd3901414d0 2024-03-16T10:11:07.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd395686640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd3900afdf0 con 0x7fd390067220 2024-03-16T10:11:07.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd38e7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3900b4e50 0x7fd39013cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd38e7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3900b4e50 0x7fd39013cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33958/0 (socket says 172.21.15.17:33958) 2024-03-16T10:11:07.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd38e7fc640 1 -- 172.21.15.17:0/2241395189 learned_addr learned my addr 172.21.15.17:0/2241395189 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:07.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd38effd640 1 --2- 172.21.15.17:0/2241395189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd390067220 0x7fd39013e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd38f7fe640 1 --2- 172.21.15.17:0/2241395189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3901414d0 0x7fd390138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd38e7fc640 1 -- 172.21.15.17:0/2241395189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd390067220 msgr2=0x7fd39013e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd38e7fc640 1 --2- 172.21.15.17:0/2241395189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd390067220 0x7fd39013e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd38e7fc640 1 -- 172.21.15.17:0/2241395189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3901414d0 msgr2=0x7fd390138340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd38e7fc640 1 --2- 172.21.15.17:0/2241395189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3901414d0 0x7fd390138340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd38e7fc640 1 -- 172.21.15.17:0/2241395189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39006db90 con 0x7fd3900b4e50 2024-03-16T10:11:07.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd38e7fc640 1 --2- 172.21.15.17:0/2241395189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3900b4e50 0x7fd39013cd40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd37800d800 tx=0x7fd37800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:07.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd3877fe640 1 -- 172.21.15.17:0/2241395189 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37800b6c0 con 0x7fd3900b4e50 2024-03-16T10:11:07.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd3877fe640 1 -- 172.21.15.17:0/2241395189 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd378019070 con 0x7fd3900b4e50 2024-03-16T10:11:07.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd3877fe640 1 -- 172.21.15.17:0/2241395189 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37800ba60 con 0x7fd3900b4e50 2024-03-16T10:11:07.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd395686640 1 -- 172.21.15.17:0/2241395189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3900701c0 con 0x7fd3900b4e50 2024-03-16T10:11:07.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd395686640 1 -- 172.21.15.17:0/2241395189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39013a720 con 0x7fd3900b4e50 2024-03-16T10:11:07.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.221+0000 7fd395686640 1 -- 172.21.15.17:0/2241395189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd39006a130 con 0x7fd3900b4e50 2024-03-16T10:11:07.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.225+0000 7fd3877fe640 1 -- 172.21.15.17:0/2241395189 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd37800bc00 con 0x7fd3900b4e50 2024-03-16T10:11:07.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.225+0000 7fd3877fe640 1 --2- 172.21.15.17:0/2241395189 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd370067660 0x7fd370069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.225+0000 7fd3877fe640 1 -- 172.21.15.17:0/2241395189 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd378086790 con 0x7fd3900b4e50 2024-03-16T10:11:07.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.225+0000 7fd38effd640 1 --2- 172.21.15.17:0/2241395189 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd370067660 0x7fd370069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.225+0000 7fd38effd640 1 --2- 172.21.15.17:0/2241395189 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd370067660 0x7fd370069b20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd3880af000 tx=0x7fd388008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:07.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.229+0000 7fd3877fe640 1 -- 172.21.15.17:0/2241395189 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd3780574f0 con 0x7fd3900b4e50 2024-03-16T10:11:07.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.425+0000 7fd395686640 1 -- 172.21.15.17:0/2241395189 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd39013f690 con 0x7fd3900b4e50 2024-03-16T10:11:07.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.429+0000 7fd3877fe640 1 -- 172.21.15.17:0/2241395189 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7fd378056df0 con 0x7fd3900b4e50 2024-03-16T10:11:07.432 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:07.432 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1058712158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1058712158},{"type":"v1","addr":"172.21.15.174:6835","nonce":1058712158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:10:57.464714+0000","modified":"2024-03-16T10:11:00.509595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6833/3105182870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3105182870},{"type":"v1","addr":"172.21.15.17:6833","nonce":3105182870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-16T10:11:07.433 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 8 2024-03-16T10:11:07.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.429+0000 7fd395686640 1 -- 172.21.15.17:0/2241395189 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd370067660 msgr2=0x7fd370069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.429+0000 7fd395686640 1 --2- 172.21.15.17:0/2241395189 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd370067660 0x7fd370069b20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd3880af000 tx=0x7fd388008040 comp rx=0 tx=0).stop 2024-03-16T10:11:07.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.429+0000 7fd395686640 1 -- 172.21.15.17:0/2241395189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3900b4e50 msgr2=0x7fd39013cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.429+0000 7fd395686640 1 --2- 172.21.15.17:0/2241395189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3900b4e50 0x7fd39013cd40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd37800d800 tx=0x7fd37800dcc0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.429+0000 7fd395686640 1 -- 172.21.15.17:0/2241395189 shutdown_connections 2024-03-16T10:11:07.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.429+0000 7fd395686640 1 --2- 172.21.15.17:0/2241395189 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd370067660 0x7fd370069b20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.429+0000 7fd395686640 1 --2- 172.21.15.17:0/2241395189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3901414d0 0x7fd390138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.429+0000 7fd395686640 1 --2- 172.21.15.17:0/2241395189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd3900b4e50 0x7fd39013cd40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.429+0000 7fd395686640 1 --2- 172.21.15.17:0/2241395189 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd390067220 0x7fd39013e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.429+0000 7fd395686640 1 -- 172.21.15.17:0/2241395189 >> 172.21.15.17:0/2241395189 conn(0x7fd39006bd70 msgr2=0x7fd39013eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:07.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.429+0000 7fd395686640 1 -- 172.21.15.17:0/2241395189 shutdown_connections 2024-03-16T10:11:07.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.429+0000 7fd395686640 1 -- 172.21.15.17:0/2241395189 wait complete. 2024-03-16T10:11:07.451 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:07.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.565+0000 7f49d27d6640 1 Processor -- start 2024-03-16T10:11:07.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49d27d6640 1 -- start start 2024-03-16T10:11:07.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49d27d6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49cc0b48f0 0x7f49cc1487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49d27d6640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49cc073040 con 0x7f49cc0b4da0 2024-03-16T10:11:07.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49d27d6640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49cc13f990 con 0x7f49cc072c60 2024-03-16T10:11:07.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49d27d6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f49cc148e50 con 0x7f49cc0b48f0 2024-03-16T10:11:07.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49cb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49cc0b48f0 0x7f49cc1487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49d0d73640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f49cc0b4da0 0x7f49cc0b5210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56158/0 (socket says 172.21.15.17:56158) 2024-03-16T10:11:07.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49cb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f49cc0b48f0 0x7f49cc1487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33974/0 (socket says 172.21.15.17:33974) 2024-03-16T10:11:07.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49d0d73640 1 -- 172.21.15.17:0/682312710 learned_addr learned my addr 172.21.15.17:0/682312710 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:07.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49cb7fe640 1 -- 172.21.15.17:0/682312710 >> v1:172.21.15.174:6789/0 conn(0x7f49cc072c60 legacy=0x7f49cc13f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49cb7fe640 1 -- 172.21.15.17:0/682312710 >> v1:172.21.15.17:6789/0 conn(0x7f49cc0b4da0 legacy=0x7f49cc0b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49caffd640 1 -- 172.21.15.17:0/682312710 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 75351452 0 0) 0x7f49cc073040 con 0x7f49cc0b4da0 2024-03-16T10:11:07.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49cb7fe640 1 -- 172.21.15.17:0/682312710 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc149130 con 0x7f49cc0b48f0 2024-03-16T10:11:07.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49cb7fe640 1 --2- 172.21.15.17:0/682312710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0b48f0 0x7f49cc1487c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f49c00097c0 tx=0x7f49c001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc67ee9c4538158 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:07.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49caffd640 1 -- 172.21.15.17:0/682312710 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00b0070 con 0x7f49cc0b48f0 2024-03-16T10:11:07.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49caffd640 1 -- 172.21.15.17:0/682312710 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c00ab600 con 0x7f49cc0b48f0 2024-03-16T10:11:07.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.569+0000 7f49caffd640 1 -- 172.21.15.17:0/682312710 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00ab9d0 con 0x7f49cc0b48f0 2024-03-16T10:11:07.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 -- 172.21.15.17:0/682312710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0b48f0 msgr2=0x7f49cc1487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 --2- 172.21.15.17:0/682312710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0b48f0 0x7f49cc1487c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f49c00097c0 tx=0x7f49c001c380 comp rx=0 tx=0).stop 2024-03-16T10:11:07.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 -- 172.21.15.17:0/682312710 shutdown_connections 2024-03-16T10:11:07.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 --2- 172.21.15.17:0/682312710 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0b48f0 0x7f49cc1487c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 -- 172.21.15.17:0/682312710 >> 172.21.15.17:0/682312710 conn(0x7f49cc0b9a30 msgr2=0x7f49cc0b9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:07.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 -- 172.21.15.17:0/682312710 shutdown_connections 2024-03-16T10:11:07.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 -- 172.21.15.17:0/682312710 wait complete. 2024-03-16T10:11:07.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 Processor -- start 2024-03-16T10:11:07.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 -- start start 2024-03-16T10:11:07.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49cc072c60 0x7f49cc13bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0b48f0 0x7f49cc0b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49cc0b4da0 0x7f49cc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc153ae0 con 0x7f49cc0b48f0 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc153960 con 0x7f49cc0b4da0 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f49cc153c60 con 0x7f49cc072c60 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0b48f0 0x7f49cc0b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0b48f0 0x7f49cc0b8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33986/0 (socket says 172.21.15.17:33986) 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49cb7fe640 1 -- 172.21.15.17:0/3218738174 learned_addr learned my addr 172.21.15.17:0/3218738174 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49cbfff640 1 --2- 172.21.15.17:0/3218738174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49cc072c60 0x7f49cc13bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49cb7fe640 1 -- 172.21.15.17:0/3218738174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49cc072c60 msgr2=0x7f49cc13bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49cb7fe640 1 --2- 172.21.15.17:0/3218738174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49cc072c60 0x7f49cc13bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49cb7fe640 1 -- 172.21.15.17:0/3218738174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49cc0b4da0 msgr2=0x7f49cc0b7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49cb7fe640 1 --2- 172.21.15.17:0/3218738174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49cc0b4da0 0x7f49cc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49cb7fe640 1 -- 172.21.15.17:0/3218738174 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc148800 con 0x7f49cc0b48f0 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49cb7fe640 1 --2- 172.21.15.17:0/3218738174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0b48f0 0x7f49cc0b8fb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f49c0018f90 tx=0x7f49c001abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:07.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49c8ff9640 1 -- 172.21.15.17:0/3218738174 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00b0050 con 0x7f49cc0b48f0 2024-03-16T10:11:07.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49c8ff9640 1 -- 172.21.15.17:0/3218738174 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49c00bc070 con 0x7f49cc0b48f0 2024-03-16T10:11:07.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 -- 172.21.15.17:0/3218738174 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49cc1e38a0 con 0x7f49cc0b48f0 2024-03-16T10:11:07.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49c8ff9640 1 -- 172.21.15.17:0/3218738174 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c00aac00 con 0x7f49cc0b48f0 2024-03-16T10:11:07.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.573+0000 7f49d27d6640 1 -- 172.21.15.17:0/3218738174 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49cc1e3e60 con 0x7f49cc0b48f0 2024-03-16T10:11:07.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.577+0000 7f49d27d6640 1 -- 172.21.15.17:0/3218738174 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49cc06a130 con 0x7f49cc0b48f0 2024-03-16T10:11:07.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.577+0000 7f49c8ff9640 1 -- 172.21.15.17:0/3218738174 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f49c00c1460 con 0x7f49cc0b48f0 2024-03-16T10:11:07.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.577+0000 7f49c8ff9640 1 --2- 172.21.15.17:0/3218738174 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f49b8067660 0x7f49b8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.577+0000 7f49c8ff9640 1 -- 172.21.15.17:0/3218738174 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f49c0120350 con 0x7f49cc0b48f0 2024-03-16T10:11:07.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.577+0000 7f49cbfff640 1 --2- 172.21.15.17:0/3218738174 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f49b8067660 0x7f49b8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.577+0000 7f49cbfff640 1 --2- 172.21.15.17:0/3218738174 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f49b8067660 0x7f49b8069b20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49bc000f70 tx=0x7f49bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:07.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.581+0000 7f49c8ff9640 1 -- 172.21.15.17:0/3218738174 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f49c0016090 con 0x7f49cc0b48f0 2024-03-16T10:11:07.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.737+0000 7f49d27d6640 1 -- 172.21.15.17:0/3218738174 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49cc13ec40 con 0x7f49cc0b48f0 2024-03-16T10:11:07.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.741+0000 7f49c8ff9640 1 -- 172.21.15.17:0/3218738174 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7f49c00f1180 con 0x7f49cc0b48f0 2024-03-16T10:11:07.746 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:07.746 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:07.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.745+0000 7f49d27d6640 1 -- 172.21.15.17:0/3218738174 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f49b8067660 msgr2=0x7f49b8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.745+0000 7f49d27d6640 1 --2- 172.21.15.17:0/3218738174 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f49b8067660 0x7f49b8069b20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49bc000f70 tx=0x7f49bc009040 comp rx=0 tx=0).stop 2024-03-16T10:11:07.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.745+0000 7f49d27d6640 1 -- 172.21.15.17:0/3218738174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0b48f0 msgr2=0x7f49cc0b8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.745+0000 7f49d27d6640 1 --2- 172.21.15.17:0/3218738174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0b48f0 0x7f49cc0b8fb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f49c0018f90 tx=0x7f49c001abc0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.745+0000 7f49d27d6640 1 -- 172.21.15.17:0/3218738174 shutdown_connections 2024-03-16T10:11:07.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.745+0000 7f49d27d6640 1 --2- 172.21.15.17:0/3218738174 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f49b8067660 0x7f49b8069b20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.745+0000 7f49d27d6640 1 --2- 172.21.15.17:0/3218738174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49cc0b4da0 0x7f49cc0b7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.745+0000 7f49d27d6640 1 --2- 172.21.15.17:0/3218738174 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49cc0b48f0 0x7f49cc0b8fb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.745+0000 7f49d27d6640 1 --2- 172.21.15.17:0/3218738174 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f49cc072c60 0x7f49cc13bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.745+0000 7f49d27d6640 1 -- 172.21.15.17:0/3218738174 >> 172.21.15.17:0/3218738174 conn(0x7f49cc0b9a30 msgr2=0x7f49cc06b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:07.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.745+0000 7f49d27d6640 1 -- 172.21.15.17:0/3218738174 shutdown_connections 2024-03-16T10:11:07.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.745+0000 7f49d27d6640 1 -- 172.21.15.17:0/3218738174 wait complete. 2024-03-16T10:11:07.763 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:10:55.723734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:10:56.459894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:11:07.764 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:11:07.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c8be10640 1 Processor -- start 2024-03-16T10:11:07.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c8be10640 1 -- start start 2024-03-16T10:11:07.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c8be10640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c84073070 0x7f1c840b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c8be10640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c840b6fd0 con 0x7f1c840b7770 2024-03-16T10:11:07.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c8be10640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c840b64f0 con 0x7f1c840b5620 2024-03-16T10:11:07.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c8be10640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1c840b67d0 con 0x7f1c84073070 2024-03-16T10:11:07.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c89bac640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1c840b7770 0x7f1c84145200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56162/0 (socket says 172.21.15.17:56162) 2024-03-16T10:11:07.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c89bac640 1 -- 172.21.15.17:0/808396672 learned_addr learned my addr 172.21.15.17:0/808396672 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:07.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c893ab640 1 --2- 172.21.15.17:0/808396672 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c84073070 0x7f1c840b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 -- 172.21.15.17:0/808396672 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 817093115 0 0) 0x7f1c840b6fd0 con 0x7f1c840b7770 2024-03-16T10:11:07.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 -- 172.21.15.17:0/808396672 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c70003530 con 0x7f1c840b7770 2024-03-16T10:11:07.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 -- 172.21.15.17:0/808396672 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3106488084 0 0) 0x7f1c70003530 con 0x7f1c840b7770 2024-03-16T10:11:07.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 -- 172.21.15.17:0/808396672 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c84073070 msgr2=0x7f1c840b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:07.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 --2- 172.21.15.17:0/808396672 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c84073070 0x7f1c840b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 -- 172.21.15.17:0/808396672 >> v1:172.21.15.174:6789/0 conn(0x7f1c840b5620 legacy=0x7f1c8414faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 -- 172.21.15.17:0/808396672 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c840b6ab0 con 0x7f1c840b7770 2024-03-16T10:11:07.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 -- 172.21.15.17:0/808396672 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f1c80001680 con 0x7f1c840b7770 2024-03-16T10:11:07.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 -- 172.21.15.17:0/808396672 >> v1:172.21.15.17:6789/0 conn(0x7f1c840b7770 legacy=0x7f1c84145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 --2- 172.21.15.17:0/808396672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c700039f0 0x7f1c70003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 --2- 172.21.15.17:0/808396672 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c84073070 0x7f1c70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 --2- 172.21.15.17:0/808396672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c70004d50 0x7f1c700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 -- 172.21.15.17:0/808396672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1c840b67d0 con 0x7f1c70004d50 2024-03-16T10:11:07.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 -- 172.21.15.17:0/808396672 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c7000bb90 con 0x7f1c700039f0 2024-03-16T10:11:07.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c88baa640 1 -- 172.21.15.17:0/808396672 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1c7000bd90 con 0x7f1c84073070 2024-03-16T10:11:07.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c8a3ad640 1 --2- 172.21.15.17:0/808396672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c70004d50 0x7f1c700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c893ab640 1 --2- 172.21.15.17:0/808396672 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c84073070 0x7f1c70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c89bac640 1 --2- 172.21.15.17:0/808396672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c700039f0 0x7f1c70003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c8a3ad640 1 -- 172.21.15.17:0/808396672 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c84073070 msgr2=0x7f1c70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:07.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c8a3ad640 1 --2- 172.21.15.17:0/808396672 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c84073070 0x7f1c70004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c8a3ad640 1 -- 172.21.15.17:0/808396672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c700039f0 msgr2=0x7f1c70003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:07.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c8a3ad640 1 --2- 172.21.15.17:0/808396672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c700039f0 0x7f1c70003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c8a3ad640 1 -- 172.21.15.17:0/808396672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c70003530 con 0x7f1c70004d50 2024-03-16T10:11:07.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.877+0000 7f1c893ab640 1 --2- 172.21.15.17:0/808396672 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c84073070 0x7f1c70004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:07.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.881+0000 7f1c8a3ad640 1 --2- 172.21.15.17:0/808396672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c70004d50 0x7f1c700051d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c001e70 tx=0x7f1c7c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf88fce1b730d3f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:07.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.881+0000 7f1c88baa640 1 -- 172.21.15.17:0/808396672 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c001a90 con 0x7f1c70004d50 2024-03-16T10:11:07.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.881+0000 7f1c88baa640 1 -- 172.21.15.17:0/808396672 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c7c003b80 con 0x7f1c70004d50 2024-03-16T10:11:07.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.881+0000 7f1c88baa640 1 -- 172.21.15.17:0/808396672 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c005080 con 0x7f1c70004d50 2024-03-16T10:11:07.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.881+0000 7f1c8be10640 1 -- 172.21.15.17:0/808396672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c70004d50 msgr2=0x7f1c700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.881+0000 7f1c8be10640 1 --2- 172.21.15.17:0/808396672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c70004d50 0x7f1c700051d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c001e70 tx=0x7f1c7c01c780 comp rx=0 tx=0).stop 2024-03-16T10:11:07.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.881+0000 7f1c8a3ad640 1 -- 172.21.15.17:0/808396672 reap_dead start 2024-03-16T10:11:07.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.881+0000 7f1c8be10640 1 -- 172.21.15.17:0/808396672 shutdown_connections 2024-03-16T10:11:07.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.881+0000 7f1c8be10640 1 -- 172.21.15.17:0/808396672 >> 172.21.15.17:0/808396672 conn(0x7f1c8406b290 msgr2=0x7f1c8406b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:07.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.881+0000 7f1c8be10640 1 -- 172.21.15.17:0/808396672 shutdown_connections 2024-03-16T10:11:07.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.881+0000 7f1c8be10640 1 -- 172.21.15.17:0/808396672 wait complete. 2024-03-16T10:11:07.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8be10640 1 Processor -- start 2024-03-16T10:11:07.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8be10640 1 -- start start 2024-03-16T10:11:07.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8be10640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c70004d50 0x7f1c841e0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8be10640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c8414afe0 0x7f1c8414b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8be10640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c8414e400 0x7f1c8414e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8be10640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84151340 con 0x7f1c8414e400 2024-03-16T10:11:07.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8be10640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841511c0 con 0x7f1c70004d50 2024-03-16T10:11:07.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8be10640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1c841514c0 con 0x7f1c8414afe0 2024-03-16T10:11:07.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8a3ad640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c8414e400 0x7f1c8414e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8a3ad640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c8414e400 0x7f1c8414e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33992/0 (socket says 172.21.15.17:33992) 2024-03-16T10:11:07.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8a3ad640 1 -- 172.21.15.17:0/2029932182 learned_addr learned my addr 172.21.15.17:0/2029932182 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:07.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8a3ad640 1 -- 172.21.15.17:0/2029932182 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c8414afe0 msgr2=0x7f1c8414b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8a3ad640 1 --2- 172.21.15.17:0/2029932182 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c8414afe0 0x7f1c8414b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8a3ad640 1 -- 172.21.15.17:0/2029932182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c70004d50 msgr2=0x7f1c841e0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:07.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8a3ad640 1 --2- 172.21.15.17:0/2029932182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c70004d50 0x7f1c841e0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:07.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8a3ad640 1 -- 172.21.15.17:0/2029932182 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c84146a80 con 0x7f1c8414e400 2024-03-16T10:11:07.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8a3ad640 1 --2- 172.21.15.17:0/2029932182 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c8414e400 0x7f1c8414e8b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c01ccf0 tx=0x7f1c70005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:07.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c7a7fc640 1 -- 172.21.15.17:0/2029932182 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0bc070 con 0x7f1c8414e400 2024-03-16T10:11:07.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c7a7fc640 1 -- 172.21.15.17:0/2029932182 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c7c005640 con 0x7f1c8414e400 2024-03-16T10:11:07.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8be10640 1 -- 172.21.15.17:0/2029932182 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c841e1740 con 0x7f1c8414e400 2024-03-16T10:11:07.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c7a7fc640 1 -- 172.21.15.17:0/2029932182 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c001a90 con 0x7f1c8414e400 2024-03-16T10:11:07.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8be10640 1 -- 172.21.15.17:0/2029932182 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c8414f570 con 0x7f1c8414e400 2024-03-16T10:11:07.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c8be10640 1 -- 172.21.15.17:0/2029932182 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c8406a130 con 0x7f1c8414e400 2024-03-16T10:11:07.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.885+0000 7f1c7a7fc640 1 -- 172.21.15.17:0/2029932182 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f1c7c0b1040 con 0x7f1c8414e400 2024-03-16T10:11:07.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.889+0000 7f1c7a7fc640 1 --2- 172.21.15.17:0/2029932182 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1c68067690 0x7f1c68069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:07.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.889+0000 7f1c7a7fc640 1 -- 172.21.15.17:0/2029932182 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1c7c11f280 con 0x7f1c8414e400 2024-03-16T10:11:07.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.889+0000 7f1c89bac640 1 --2- 172.21.15.17:0/2029932182 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1c68067690 0x7f1c68069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:07.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.889+0000 7f1c89bac640 1 --2- 172.21.15.17:0/2029932182 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1c68067690 0x7f1c68069b50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c80000fc0 tx=0x7f1c80001090 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:07.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:07.893+0000 7f1c7a7fc640 1 -- 172.21.15.17:0/2029932182 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1c7c0ad020 con 0x7f1c8414e400 2024-03-16T10:11:08.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.053+0000 7f1c8be10640 1 -- 172.21.15.17:0/2029932182 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1c840b7e10 con 0x7f1c8414e400 2024-03-16T10:11:08.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.057+0000 7f1c7a7fc640 1 -- 172.21.15.17:0/2029932182 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f1c7c016490 con 0x7f1c8414e400 2024-03-16T10:11:08.060 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:11:08.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.057+0000 7f1c8be10640 1 -- 172.21.15.17:0/2029932182 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1c68067690 msgr2=0x7f1c68069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.057+0000 7f1c8be10640 1 --2- 172.21.15.17:0/2029932182 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1c68067690 0x7f1c68069b50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1c80000fc0 tx=0x7f1c80001090 comp rx=0 tx=0).stop 2024-03-16T10:11:08.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.057+0000 7f1c8be10640 1 -- 172.21.15.17:0/2029932182 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c8414e400 msgr2=0x7f1c8414e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.057+0000 7f1c8be10640 1 --2- 172.21.15.17:0/2029932182 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c8414e400 0x7f1c8414e8b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c01ccf0 tx=0x7f1c70005840 comp rx=0 tx=0).stop 2024-03-16T10:11:08.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.057+0000 7f1c8be10640 1 -- 172.21.15.17:0/2029932182 shutdown_connections 2024-03-16T10:11:08.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.057+0000 7f1c8be10640 1 --2- 172.21.15.17:0/2029932182 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1c68067690 0x7f1c68069b50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.057+0000 7f1c8be10640 1 --2- 172.21.15.17:0/2029932182 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1c8414e400 0x7f1c8414e8b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.057+0000 7f1c8be10640 1 --2- 172.21.15.17:0/2029932182 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1c8414afe0 0x7f1c8414b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.057+0000 7f1c8be10640 1 --2- 172.21.15.17:0/2029932182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c70004d50 0x7f1c841e0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.057+0000 7f1c8be10640 1 -- 172.21.15.17:0/2029932182 >> 172.21.15.17:0/2029932182 conn(0x7f1c8406b290 msgr2=0x7f1c840b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:08.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.057+0000 7f1c8be10640 1 -- 172.21.15.17:0/2029932182 shutdown_connections 2024-03-16T10:11:08.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.057+0000 7f1c8be10640 1 -- 172.21.15.17:0/2029932182 wait complete. 2024-03-16T10:11:08.079 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:11:08.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd99f22640 1 Processor -- start 2024-03-16T10:11:08.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd99f22640 1 -- start start 2024-03-16T10:11:08.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd99f22640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd940aeaf0 0x7efd94071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd99f22640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7efd94072410 0x7efd9414e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd99f22640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd940aef60 con 0x7efd940b5740 2024-03-16T10:11:08.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd99f22640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7efd9406e950 con 0x7efd940aeaf0 2024-03-16T10:11:08.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd99f22640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7efd940b5b20 con 0x7efd94072410 2024-03-16T10:11:08.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd92ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd940aeaf0 0x7efd94071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd92ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efd940aeaf0 0x7efd94071ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33996/0 (socket says 172.21.15.17:33996) 2024-03-16T10:11:08.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd937fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efd940b5740 0x7efd9406ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56176/0 (socket says 172.21.15.17:56176) 2024-03-16T10:11:08.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd92ffd640 1 -- 172.21.15.17:0/2277555458 learned_addr learned my addr 172.21.15.17:0/2277555458 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:08.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd93fff640 1 --2- 172.21.15.17:0/2277555458 >> v2:172.21.15.174:3300/0 conn(0x7efd94072410 0x7efd9414e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd92ffd640 1 -- 172.21.15.17:0/2277555458 >> v2:172.21.15.174:3300/0 conn(0x7efd94072410 msgr2=0x7efd9414e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:08.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd92ffd640 1 --2- 172.21.15.17:0/2277555458 >> v2:172.21.15.174:3300/0 conn(0x7efd94072410 0x7efd9414e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd92ffd640 1 -- 172.21.15.17:0/2277555458 >> v1:172.21.15.17:6789/0 conn(0x7efd940b5740 legacy=0x7efd9406ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd92ffd640 1 -- 172.21.15.17:0/2277555458 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd940af720 con 0x7efd940aeaf0 2024-03-16T10:11:08.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.197+0000 7efd93fff640 1 --2- 172.21.15.17:0/2277555458 >> v2:172.21.15.174:3300/0 conn(0x7efd94072410 0x7efd9414e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:08.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.201+0000 7efd92ffd640 1 --2- 172.21.15.17:0/2277555458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940aeaf0 0x7efd94071ed0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd940706a0 tx=0x7efd8801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b04f3f7d85c559 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:08.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.201+0000 7efd927fc640 1 -- 172.21.15.17:0/2277555458 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd880b4070 con 0x7efd940aeaf0 2024-03-16T10:11:08.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.201+0000 7efd927fc640 1 -- 172.21.15.17:0/2277555458 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd880a79c0 con 0x7efd940aeaf0 2024-03-16T10:11:08.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.201+0000 7efd927fc640 1 -- 172.21.15.17:0/2277555458 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd880a7d90 con 0x7efd940aeaf0 2024-03-16T10:11:08.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.201+0000 7efd99f22640 1 -- 172.21.15.17:0/2277555458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940aeaf0 msgr2=0x7efd94071ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.201+0000 7efd99f22640 1 --2- 172.21.15.17:0/2277555458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940aeaf0 0x7efd94071ed0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd940706a0 tx=0x7efd8801ebf0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.201+0000 7efd99f22640 1 -- 172.21.15.17:0/2277555458 shutdown_connections 2024-03-16T10:11:08.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.201+0000 7efd99f22640 1 --2- 172.21.15.17:0/2277555458 >> v2:172.21.15.174:3300/0 conn(0x7efd94072410 0x7efd9414e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.201+0000 7efd99f22640 1 --2- 172.21.15.17:0/2277555458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940aeaf0 0x7efd94071ed0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.201+0000 7efd99f22640 1 -- 172.21.15.17:0/2277555458 >> 172.21.15.17:0/2277555458 conn(0x7efd940b65d0 msgr2=0x7efd940b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:08.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.201+0000 7efd99f22640 1 -- 172.21.15.17:0/2277555458 shutdown_connections 2024-03-16T10:11:08.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.201+0000 7efd99f22640 1 -- 172.21.15.17:0/2277555458 wait complete. 2024-03-16T10:11:08.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.201+0000 7efd99f22640 1 Processor -- start 2024-03-16T10:11:08.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.201+0000 7efd99f22640 1 -- start start 2024-03-16T10:11:08.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd99f22640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd94072410 0x7efd94143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd99f22640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd940aeaf0 0x7efd9413fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd99f22640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940b5740 0x7efd941df170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd99f22640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efd94147620 con 0x7efd940b5740 2024-03-16T10:11:08.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd99f22640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efd941477a0 con 0x7efd940aeaf0 2024-03-16T10:11:08.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd99f22640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7efd9406baa0 con 0x7efd94072410 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd93fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940b5740 0x7efd941df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd93fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940b5740 0x7efd941df170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34008/0 (socket says 172.21.15.17:34008) 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd93fff640 1 -- 172.21.15.17:0/2465173417 learned_addr learned my addr 172.21.15.17:0/2465173417 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd937fe640 1 --2- 172.21.15.17:0/2465173417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd94072410 0x7efd94143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd92ffd640 1 --2- 172.21.15.17:0/2465173417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd940aeaf0 0x7efd9413fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd93fff640 1 -- 172.21.15.17:0/2465173417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd94072410 msgr2=0x7efd94143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd93fff640 1 --2- 172.21.15.17:0/2465173417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd94072410 0x7efd94143a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd93fff640 1 -- 172.21.15.17:0/2465173417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd940aeaf0 msgr2=0x7efd9413fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd93fff640 1 --2- 172.21.15.17:0/2465173417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd940aeaf0 0x7efd9413fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd93fff640 1 -- 172.21.15.17:0/2465173417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd94142b50 con 0x7efd940b5740 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd93fff640 1 --2- 172.21.15.17:0/2465173417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940b5740 0x7efd941df170 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009fd0 tx=0x7efd8c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd7bfff640 1 -- 172.21.15.17:0/2465173417 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c01a070 con 0x7efd940b5740 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd7bfff640 1 -- 172.21.15.17:0/2465173417 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8c003bc0 con 0x7efd940b5740 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd99f22640 1 -- 172.21.15.17:0/2465173417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd94142e30 con 0x7efd940b5740 2024-03-16T10:11:08.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd7bfff640 1 -- 172.21.15.17:0/2465173417 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c004030 con 0x7efd940b5740 2024-03-16T10:11:08.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd99f22640 1 -- 172.21.15.17:0/2465173417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd94143fa0 con 0x7efd940b5740 2024-03-16T10:11:08.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd99f22640 1 -- 172.21.15.17:0/2465173417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9406a130 con 0x7efd940b5740 2024-03-16T10:11:08.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd7bfff640 1 -- 172.21.15.17:0/2465173417 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7efd8c01d070 con 0x7efd940b5740 2024-03-16T10:11:08.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd7bfff640 1 --2- 172.21.15.17:0/2465173417 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efd80067660 0x7efd80069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd7bfff640 1 -- 172.21.15.17:0/2465173417 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efd8c085b20 con 0x7efd940b5740 2024-03-16T10:11:08.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.205+0000 7efd937fe640 1 --2- 172.21.15.17:0/2465173417 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efd80067660 0x7efd80069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.209+0000 7efd937fe640 1 --2- 172.21.15.17:0/2465173417 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efd80067660 0x7efd80069b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efd84000f70 tx=0x7efd84007cf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:08.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.213+0000 7efd7bfff640 1 -- 172.21.15.17:0/2465173417 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7efd8c010070 con 0x7efd940b5740 2024-03-16T10:11:08.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.373+0000 7efd99f22640 1 -- 172.21.15.17:0/2465173417 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efd94072820 con 0x7efd940b5740 2024-03-16T10:11:08.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.373+0000 7efd7bfff640 1 -- 172.21.15.17:0/2465173417 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7efd94072820 con 0x7efd940b5740 2024-03-16T10:11:08.377 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-16T10:11:08.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.373+0000 7efd99f22640 1 -- 172.21.15.17:0/2465173417 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efd80067660 msgr2=0x7efd80069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.373+0000 7efd99f22640 1 --2- 172.21.15.17:0/2465173417 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efd80067660 0x7efd80069b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efd84000f70 tx=0x7efd84007cf0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.373+0000 7efd99f22640 1 -- 172.21.15.17:0/2465173417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940b5740 msgr2=0x7efd941df170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.373+0000 7efd99f22640 1 --2- 172.21.15.17:0/2465173417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940b5740 0x7efd941df170 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009fd0 tx=0x7efd8c00d560 comp rx=0 tx=0).stop 2024-03-16T10:11:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.373+0000 7efd99f22640 1 -- 172.21.15.17:0/2465173417 shutdown_connections 2024-03-16T10:11:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.373+0000 7efd99f22640 1 --2- 172.21.15.17:0/2465173417 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efd80067660 0x7efd80069b20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.373+0000 7efd99f22640 1 --2- 172.21.15.17:0/2465173417 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd940b5740 0x7efd941df170 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.373+0000 7efd99f22640 1 --2- 172.21.15.17:0/2465173417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd940aeaf0 0x7efd9413fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.373+0000 7efd99f22640 1 --2- 172.21.15.17:0/2465173417 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efd94072410 0x7efd94143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.373+0000 7efd99f22640 1 -- 172.21.15.17:0/2465173417 >> 172.21.15.17:0/2465173417 conn(0x7efd940b65d0 msgr2=0x7efd9406d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:08.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.377+0000 7efd99f22640 1 -- 172.21.15.17:0/2465173417 shutdown_connections 2024-03-16T10:11:08.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.377+0000 7efd99f22640 1 -- 172.21.15.17:0/2465173417 wait complete. 2024-03-16T10:11:08.396 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:11:08.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f7545e84640 1 Processor -- start 2024-03-16T10:11:08.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f7545e84640 1 -- start start 2024-03-16T10:11:08.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f7545e84640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f75400b49d0 0x7f7540150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f7545e84640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75400b6fd0 con 0x7f75400b7770 2024-03-16T10:11:08.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f7545e84640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75400b64f0 con 0x7f7540073070 2024-03-16T10:11:08.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f7545e84640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f75400b67d0 con 0x7f75400b49d0 2024-03-16T10:11:08.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753f7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f75400b7770 0x7f7540145200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56190/0 (socket says 172.21.15.17:56190) 2024-03-16T10:11:08.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753f7fe640 1 -- 172.21.15.17:0/3681559601 learned_addr learned my addr 172.21.15.17:0/3681559601 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753ffff640 1 --2- 172.21.15.17:0/3681559601 >> v2:172.21.15.174:3300/0 conn(0x7f75400b49d0 0x7f7540150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 -- 172.21.15.17:0/3681559601 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3858502044 0 0) 0x7f75400b6fd0 con 0x7f75400b7770 2024-03-16T10:11:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 -- 172.21.15.17:0/3681559601 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f752c003530 con 0x7f75400b7770 2024-03-16T10:11:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 -- 172.21.15.17:0/3681559601 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1383384182 0 0) 0x7f752c003530 con 0x7f75400b7770 2024-03-16T10:11:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 -- 172.21.15.17:0/3681559601 >> v2:172.21.15.174:3300/0 conn(0x7f75400b49d0 msgr2=0x7f7540150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:08.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 --2- 172.21.15.17:0/3681559601 >> v2:172.21.15.174:3300/0 conn(0x7f75400b49d0 0x7f7540150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 -- 172.21.15.17:0/3681559601 >> v1:172.21.15.174:6789/0 conn(0x7f7540073070 legacy=0x7f75400b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 -- 172.21.15.17:0/3681559601 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75400b6ab0 con 0x7f75400b7770 2024-03-16T10:11:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 -- 172.21.15.17:0/3681559601 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f7530001680 con 0x7f75400b7770 2024-03-16T10:11:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 -- 172.21.15.17:0/3681559601 >> v1:172.21.15.17:6789/0 conn(0x7f75400b7770 legacy=0x7f7540145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 --2- 172.21.15.17:0/3681559601 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f752c0039c0 0x7f752c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 --2- 172.21.15.17:0/3681559601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752c004810 0x7f752c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 --2- 172.21.15.17:0/3681559601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f752c0051d0 0x7f752c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 -- 172.21.15.17:0/3681559601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f75400b67d0 con 0x7f752c0051d0 2024-03-16T10:11:08.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 -- 172.21.15.17:0/3681559601 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f752c00c120 con 0x7f752c004810 2024-03-16T10:11:08.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753effd640 1 -- 172.21.15.17:0/3681559601 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f752c00c320 con 0x7f752c0039c0 2024-03-16T10:11:08.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753ffff640 1 --2- 172.21.15.17:0/3681559601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f752c0051d0 0x7f752c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f7536ffd640 1 --2- 172.21.15.17:0/3681559601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752c004810 0x7f752c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753f7fe640 1 --2- 172.21.15.17:0/3681559601 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f752c0039c0 0x7f752c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753ffff640 1 -- 172.21.15.17:0/3681559601 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f752c0039c0 msgr2=0x7f752c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:08.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753ffff640 1 --2- 172.21.15.17:0/3681559601 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f752c0039c0 0x7f752c003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753ffff640 1 -- 172.21.15.17:0/3681559601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752c004810 msgr2=0x7f752c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:08.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753ffff640 1 --2- 172.21.15.17:0/3681559601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f752c004810 0x7f752c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.509+0000 7f753ffff640 1 -- 172.21.15.17:0/3681559601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f752c003530 con 0x7f752c0051d0 2024-03-16T10:11:08.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f753ffff640 1 -- 172.21.15.17:0/3681559601 reap_dead start 2024-03-16T10:11:08.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f753ffff640 1 --2- 172.21.15.17:0/3681559601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f752c0051d0 0x7f752c00b9d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7538009fc0 tx=0x7f753801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5df5a425dd9779 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:08.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f753effd640 1 -- 172.21.15.17:0/3681559601 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538009c10 con 0x7f752c0051d0 2024-03-16T10:11:08.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f753effd640 1 -- 172.21.15.17:0/3681559601 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75380a5d10 con 0x7f752c0051d0 2024-03-16T10:11:08.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f753effd640 1 -- 172.21.15.17:0/3681559601 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538003be0 con 0x7f752c0051d0 2024-03-16T10:11:08.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 -- 172.21.15.17:0/3681559601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f752c0051d0 msgr2=0x7f752c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 --2- 172.21.15.17:0/3681559601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f752c0051d0 0x7f752c00b9d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7538009fc0 tx=0x7f753801ccc0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 -- 172.21.15.17:0/3681559601 shutdown_connections 2024-03-16T10:11:08.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 --2- 172.21.15.17:0/3681559601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f752c0051d0 0x7f752c00b9d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 -- 172.21.15.17:0/3681559601 >> 172.21.15.17:0/3681559601 conn(0x7f754006b290 msgr2=0x7f754006b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:08.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 -- 172.21.15.17:0/3681559601 shutdown_connections 2024-03-16T10:11:08.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 -- 172.21.15.17:0/3681559601 wait complete. 2024-03-16T10:11:08.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 Processor -- start 2024-03-16T10:11:08.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 -- start start 2024-03-16T10:11:08.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f752c0051d0 0x7f75401d0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75401ddee0 0x7f75400bb380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75400a9ba0 0x7f75400b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f75400b5620 con 0x7f75401ddee0 2024-03-16T10:11:08.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f75400b7b50 con 0x7f75400a9ba0 2024-03-16T10:11:08.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7545e84640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f75400b57a0 con 0x7f752c0051d0 2024-03-16T10:11:08.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7536ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75401ddee0 0x7f75400bb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7536ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75401ddee0 0x7f75400bb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34040/0 (socket says 172.21.15.17:34040) 2024-03-16T10:11:08.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7536ffd640 1 -- 172.21.15.17:0/391195140 learned_addr learned my addr 172.21.15.17:0/391195140 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:08.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f753ffff640 1 --2- 172.21.15.17:0/391195140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75400a9ba0 0x7f75400b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f753f7fe640 1 --2- 172.21.15.17:0/391195140 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f752c0051d0 0x7f75401d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7536ffd640 1 -- 172.21.15.17:0/391195140 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f752c0051d0 msgr2=0x7f75401d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7536ffd640 1 --2- 172.21.15.17:0/391195140 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f752c0051d0 0x7f75401d0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7536ffd640 1 -- 172.21.15.17:0/391195140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75400a9ba0 msgr2=0x7f75400b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7536ffd640 1 --2- 172.21.15.17:0/391195140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75400a9ba0 0x7f75400b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7536ffd640 1 -- 172.21.15.17:0/391195140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75401cf790 con 0x7f75401ddee0 2024-03-16T10:11:08.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.513+0000 7f7536ffd640 1 --2- 172.21.15.17:0/391195140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75401ddee0 0x7f75400bb380 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7528010f70 tx=0x7f752800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:08.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.517+0000 7f753cff9640 1 -- 172.21.15.17:0/391195140 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528018020 con 0x7f75401ddee0 2024-03-16T10:11:08.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.517+0000 7f753cff9640 1 -- 172.21.15.17:0/391195140 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7528005070 con 0x7f75401ddee0 2024-03-16T10:11:08.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.517+0000 7f7545e84640 1 -- 172.21.15.17:0/391195140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75401cfa70 con 0x7f75401ddee0 2024-03-16T10:11:08.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.517+0000 7f753cff9640 1 -- 172.21.15.17:0/391195140 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528005460 con 0x7f75401ddee0 2024-03-16T10:11:08.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.517+0000 7f7545e84640 1 -- 172.21.15.17:0/391195140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75401cebf0 con 0x7f75401ddee0 2024-03-16T10:11:08.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.517+0000 7f7545e84640 1 -- 172.21.15.17:0/391195140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754006a130 con 0x7f75401ddee0 2024-03-16T10:11:08.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.517+0000 7f753cff9640 1 -- 172.21.15.17:0/391195140 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f752801b020 con 0x7f75401ddee0 2024-03-16T10:11:08.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.517+0000 7f753cff9640 1 --2- 172.21.15.17:0/391195140 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7520067660 0x7f7520069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.517+0000 7f753cff9640 1 -- 172.21.15.17:0/391195140 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f75280891f0 con 0x7f75401ddee0 2024-03-16T10:11:08.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.517+0000 7f753f7fe640 1 --2- 172.21.15.17:0/391195140 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7520067660 0x7f7520069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.517+0000 7f753f7fe640 1 --2- 172.21.15.17:0/391195140 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7520067660 0x7f7520069b20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7530001930 tx=0x7f7530009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:08.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.521+0000 7f753cff9640 1 -- 172.21.15.17:0/391195140 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7528022020 con 0x7f75401ddee0 2024-03-16T10:11:08.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.681+0000 7f7545e84640 1 -- 172.21.15.17:0/391195140 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f75400b7e10 con 0x7f75401ddee0 2024-03-16T10:11:08.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.681+0000 7f753cff9640 1 -- 172.21.15.17:0/391195140 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f7528059e60 con 0x7f75401ddee0 2024-03-16T10:11:08.685 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-16T10:11:08.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.681+0000 7f7545e84640 1 -- 172.21.15.17:0/391195140 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7520067660 msgr2=0x7f7520069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.681+0000 7f7545e84640 1 --2- 172.21.15.17:0/391195140 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7520067660 0x7f7520069b20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7530001930 tx=0x7f7530009040 comp rx=0 tx=0).stop 2024-03-16T10:11:08.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.681+0000 7f7545e84640 1 -- 172.21.15.17:0/391195140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75401ddee0 msgr2=0x7f75400bb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.681+0000 7f7545e84640 1 --2- 172.21.15.17:0/391195140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75401ddee0 0x7f75400bb380 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7528010f70 tx=0x7f752800e530 comp rx=0 tx=0).stop 2024-03-16T10:11:08.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.681+0000 7f7545e84640 1 -- 172.21.15.17:0/391195140 shutdown_connections 2024-03-16T10:11:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.681+0000 7f7545e84640 1 --2- 172.21.15.17:0/391195140 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7520067660 0x7f7520069b20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.681+0000 7f7545e84640 1 --2- 172.21.15.17:0/391195140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75400a9ba0 0x7f75400b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.681+0000 7f7545e84640 1 --2- 172.21.15.17:0/391195140 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75401ddee0 0x7f75400bb380 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.681+0000 7f7545e84640 1 --2- 172.21.15.17:0/391195140 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f752c0051d0 0x7f75401d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.681+0000 7f7545e84640 1 -- 172.21.15.17:0/391195140 >> 172.21.15.17:0/391195140 conn(0x7f754006b290 msgr2=0x7f75401ccc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:08.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.681+0000 7f7545e84640 1 -- 172.21.15.17:0/391195140 shutdown_connections 2024-03-16T10:11:08.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.681+0000 7f7545e84640 1 -- 172.21.15.17:0/391195140 wait complete. 2024-03-16T10:11:08.704 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:11:08.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.821+0000 7f03a2ea0640 1 Processor -- start 2024-03-16T10:11:08.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.821+0000 7f03a2ea0640 1 -- start start 2024-03-16T10:11:08.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f03a2ea0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f039c0b7650 0x7f039c1412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f03a2ea0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f039c141c50 con 0x7f039c0b4650 2024-03-16T10:11:08.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f03a2ea0640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f039c0b4a30 con 0x7f039c141820 2024-03-16T10:11:08.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f03a2ea0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f039c14abe0 con 0x7f039c0b7650 2024-03-16T10:11:08.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f03a0c3c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f039c0b4650 0x7f039c14b3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:56192/0 (socket says 172.21.15.17:56192) 2024-03-16T10:11:08.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f03a0c3c640 1 -- 172.21.15.17:0/2434685190 learned_addr learned my addr 172.21.15.17:0/2434685190 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:08.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039bfff640 1 --2- 172.21.15.17:0/2434685190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f039c0b7650 0x7f039c1412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 -- 172.21.15.17:0/2434685190 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 296074384 0 0) 0x7f039c141c50 con 0x7f039c0b4650 2024-03-16T10:11:08.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 -- 172.21.15.17:0/2434685190 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0384003530 con 0x7f039c0b4650 2024-03-16T10:11:08.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 -- 172.21.15.17:0/2434685190 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1608112596 0 0) 0x7f0384003530 con 0x7f039c0b4650 2024-03-16T10:11:08.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 -- 172.21.15.17:0/2434685190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f039c0b7650 msgr2=0x7f039c1412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:08.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 --2- 172.21.15.17:0/2434685190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f039c0b7650 0x7f039c1412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 -- 172.21.15.17:0/2434685190 >> v1:172.21.15.174:6789/0 conn(0x7f039c141820 legacy=0x7f039c151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 -- 172.21.15.17:0/2434685190 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c14aec0 con 0x7f039c0b4650 2024-03-16T10:11:08.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 -- 172.21.15.17:0/2434685190 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f0390001680 con 0x7f039c0b4650 2024-03-16T10:11:08.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 -- 172.21.15.17:0/2434685190 >> v1:172.21.15.17:6789/0 conn(0x7f039c0b4650 legacy=0x7f039c14b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 --2- 172.21.15.17:0/2434685190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03840039f0 0x7f0384003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 --2- 172.21.15.17:0/2434685190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f039c0b7650 0x7f0384004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 --2- 172.21.15.17:0/2434685190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0384004d50 0x7f03840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 -- 172.21.15.17:0/2434685190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f039c14abe0 con 0x7f0384004d50 2024-03-16T10:11:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 -- 172.21.15.17:0/2434685190 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f038400ba80 con 0x7f03840039f0 2024-03-16T10:11:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039b7fe640 1 -- 172.21.15.17:0/2434685190 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f038400bc80 con 0x7f039c0b7650 2024-03-16T10:11:08.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f03a143d640 1 --2- 172.21.15.17:0/2434685190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0384004d50 0x7f03840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f039bfff640 1 --2- 172.21.15.17:0/2434685190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f039c0b7650 0x7f0384004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f03a0c3c640 1 --2- 172.21.15.17:0/2434685190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03840039f0 0x7f0384003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f03a143d640 1 -- 172.21.15.17:0/2434685190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f039c0b7650 msgr2=0x7f0384004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:08.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f03a143d640 1 --2- 172.21.15.17:0/2434685190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f039c0b7650 0x7f0384004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f03a143d640 1 -- 172.21.15.17:0/2434685190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03840039f0 msgr2=0x7f0384003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f03a143d640 1 --2- 172.21.15.17:0/2434685190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03840039f0 0x7f0384003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.825+0000 7f03a143d640 1 -- 172.21.15.17:0/2434685190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0384003530 con 0x7f0384004d50 2024-03-16T10:11:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.829+0000 7f03a143d640 1 --2- 172.21.15.17:0/2434685190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0384004d50 0x7f03840051d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0394002010 tx=0x7f039401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a3a39ba80939b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.829+0000 7f039b7fe640 1 -- 172.21.15.17:0/2434685190 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0394001c10 con 0x7f0384004d50 2024-03-16T10:11:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.829+0000 7f039b7fe640 1 -- 172.21.15.17:0/2434685190 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03940a8640 con 0x7f0384004d50 2024-03-16T10:11:08.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.829+0000 7f039b7fe640 1 -- 172.21.15.17:0/2434685190 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03940a8a30 con 0x7f0384004d50 2024-03-16T10:11:08.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.829+0000 7f03a2ea0640 1 -- 172.21.15.17:0/2434685190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0384004d50 msgr2=0x7f03840051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.829+0000 7f03a2ea0640 1 --2- 172.21.15.17:0/2434685190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0384004d50 0x7f03840051d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0394002010 tx=0x7f039401c8e0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.829+0000 7f03a143d640 1 -- 172.21.15.17:0/2434685190 reap_dead start 2024-03-16T10:11:08.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.829+0000 7f03a2ea0640 1 -- 172.21.15.17:0/2434685190 shutdown_connections 2024-03-16T10:11:08.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.829+0000 7f03a2ea0640 1 -- 172.21.15.17:0/2434685190 >> 172.21.15.17:0/2434685190 conn(0x7f039c0bccd0 msgr2=0x7f039c0bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:08.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.829+0000 7f03a2ea0640 1 -- 172.21.15.17:0/2434685190 shutdown_connections 2024-03-16T10:11:08.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.829+0000 7f03a2ea0640 1 -- 172.21.15.17:0/2434685190 wait complete. 2024-03-16T10:11:08.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.829+0000 7f03a2ea0640 1 Processor -- start 2024-03-16T10:11:08.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.829+0000 7f03a2ea0640 1 -- start start 2024-03-16T10:11:08.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03a2ea0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f039c151fd0 0x7f039c1523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03a2ea0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c1e0700 0x7f039c1e0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03a2ea0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f039c1e3640 0x7f039c1e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03a2ea0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f039c0b5b70 con 0x7f039c1e0700 2024-03-16T10:11:08.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03a2ea0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f039c0b59f0 con 0x7f039c151fd0 2024-03-16T10:11:08.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03a2ea0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f039c153d00 con 0x7f039c1e3640 2024-03-16T10:11:08.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f039bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c1e0700 0x7f039c1e0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f039bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c1e0700 0x7f039c1e0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34058/0 (socket says 172.21.15.17:34058) 2024-03-16T10:11:08.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f039bfff640 1 -- 172.21.15.17:0/4031626690 learned_addr learned my addr 172.21.15.17:0/4031626690 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:08.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03a143d640 1 --2- 172.21.15.17:0/4031626690 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f039c1e3640 0x7f039c1e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03a0c3c640 1 --2- 172.21.15.17:0/4031626690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f039c151fd0 0x7f039c1523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f039bfff640 1 -- 172.21.15.17:0/4031626690 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f039c1e3640 msgr2=0x7f039c1e3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f039bfff640 1 --2- 172.21.15.17:0/4031626690 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f039c1e3640 0x7f039c1e3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f039bfff640 1 -- 172.21.15.17:0/4031626690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f039c151fd0 msgr2=0x7f039c1523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:08.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f039bfff640 1 --2- 172.21.15.17:0/4031626690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f039c151fd0 0x7f039c1523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:08.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f039bfff640 1 -- 172.21.15.17:0/4031626690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c151260 con 0x7f039c1e0700 2024-03-16T10:11:08.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f039bfff640 1 --2- 172.21.15.17:0/4031626690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c1e0700 0x7f039c1e0b60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f038c0115c0 tx=0x7f038c011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:08.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03997fa640 1 -- 172.21.15.17:0/4031626690 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c004220 con 0x7f039c1e0700 2024-03-16T10:11:08.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03997fa640 1 -- 172.21.15.17:0/4031626690 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f038c0043c0 con 0x7f039c1e0700 2024-03-16T10:11:08.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03a2ea0640 1 -- 172.21.15.17:0/4031626690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c1424a0 con 0x7f039c1e0700 2024-03-16T10:11:08.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03997fa640 1 -- 172.21.15.17:0/4031626690 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c00f8f0 con 0x7f039c1e0700 2024-03-16T10:11:08.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03a2ea0640 1 -- 172.21.15.17:0/4031626690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c142780 con 0x7f039c1e0700 2024-03-16T10:11:08.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03a2ea0640 1 -- 172.21.15.17:0/4031626690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039c068360 con 0x7f039c1e0700 2024-03-16T10:11:08.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03997fa640 1 -- 172.21.15.17:0/4031626690 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f038c00fa90 con 0x7f039c1e0700 2024-03-16T10:11:08.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03997fa640 1 --2- 172.21.15.17:0/4031626690 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0388067690 0x7f0388069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:08.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.833+0000 7f03997fa640 1 -- 172.21.15.17:0/4031626690 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f038c086190 con 0x7f039c1e0700 2024-03-16T10:11:08.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.837+0000 7f03a0c3c640 1 --2- 172.21.15.17:0/4031626690 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0388067690 0x7f0388069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:08.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.837+0000 7f03a0c3c640 1 --2- 172.21.15.17:0/4031626690 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0388067690 0x7f0388069b50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0390001ec0 tx=0x7f0390009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:08.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:08.837+0000 7f03997fa640 1 -- 172.21.15.17:0/4031626690 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f038c019030 con 0x7f039c1e0700 2024-03-16T10:11:09.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.033+0000 7f03a2ea0640 1 -- 172.21.15.17:0/4031626690 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f039c0bb130 con 0x7f039c1e0700 2024-03-16T10:11:09.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.033+0000 7f03997fa640 1 -- 172.21.15.17:0/4031626690 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7f038c056f80 con 0x7f039c1e0700 2024-03-16T10:11:09.036 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:09.036 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1058712158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1058712158},{"type":"v1","addr":"172.21.15.174:6835","nonce":1058712158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:10:57.464714+0000","modified":"2024-03-16T10:11:00.509595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6833/3105182870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3105182870},{"type":"v1","addr":"172.21.15.17:6833","nonce":3105182870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-16T10:11:09.036 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 8 2024-03-16T10:11:09.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.033+0000 7f03a2ea0640 1 -- 172.21.15.17:0/4031626690 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0388067690 msgr2=0x7f0388069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:09.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.033+0000 7f03a2ea0640 1 --2- 172.21.15.17:0/4031626690 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0388067690 0x7f0388069b50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0390001ec0 tx=0x7f0390009040 comp rx=0 tx=0).stop 2024-03-16T10:11:09.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.033+0000 7f03a2ea0640 1 -- 172.21.15.17:0/4031626690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c1e0700 msgr2=0x7f039c1e0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:09.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.033+0000 7f03a2ea0640 1 --2- 172.21.15.17:0/4031626690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c1e0700 0x7f039c1e0b60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f038c0115c0 tx=0x7f038c011a80 comp rx=0 tx=0).stop 2024-03-16T10:11:09.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.033+0000 7f03a2ea0640 1 -- 172.21.15.17:0/4031626690 shutdown_connections 2024-03-16T10:11:09.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.033+0000 7f03a2ea0640 1 --2- 172.21.15.17:0/4031626690 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0388067690 0x7f0388069b50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.033+0000 7f03a2ea0640 1 --2- 172.21.15.17:0/4031626690 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f039c1e3640 0x7f039c1e3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.033+0000 7f03a2ea0640 1 --2- 172.21.15.17:0/4031626690 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f039c1e0700 0x7f039c1e0b60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.033+0000 7f03a2ea0640 1 --2- 172.21.15.17:0/4031626690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f039c151fd0 0x7f039c1523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.033+0000 7f03a2ea0640 1 -- 172.21.15.17:0/4031626690 >> 172.21.15.17:0/4031626690 conn(0x7f039c0bccd0 msgr2=0x7f039c13db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:09.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.033+0000 7f03a2ea0640 1 -- 172.21.15.17:0/4031626690 shutdown_connections 2024-03-16T10:11:09.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.033+0000 7f03a2ea0640 1 -- 172.21.15.17:0/4031626690 wait complete. 2024-03-16T10:11:09.053 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:09.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.165+0000 7fd02c153640 1 Processor -- start 2024-03-16T10:11:09.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.165+0000 7fd02c153640 1 -- start start 2024-03-16T10:11:09.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd02c153640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0240b59e0 0x7fd02414a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:09.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd02c153640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd024073210 0x7fd02406e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:09.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd02c153640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0240b5dc0 con 0x7fd02409f5b0 2024-03-16T10:11:09.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd02c153640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd02409fa40 con 0x7fd024073210 2024-03-16T10:11:09.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd02c153640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd02409fd20 con 0x7fd0240b59e0 2024-03-16T10:11:09.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd02a6f0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd024073210 0x7fd02406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:09.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd02a6f0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd024073210 0x7fd02406e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55510/0 (socket says 172.21.15.17:55510) 2024-03-16T10:11:09.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd02a6f0640 1 -- 172.21.15.17:0/2414352258 learned_addr learned my addr 172.21.15.17:0/2414352258 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:09.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd0296ee640 1 --1- 172.21.15.17:0/2414352258 >> v1:172.21.15.17:6789/0 conn(0x7fd02409f5b0 0x7fd0240b0690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48922/0 (socket says 172.21.15.17:48922) 2024-03-16T10:11:09.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd029eef640 1 --2- 172.21.15.17:0/2414352258 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0240b59e0 0x7fd02414a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:09.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd02a6f0640 1 -- 172.21.15.17:0/2414352258 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0240b59e0 msgr2=0x7fd02414a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:09.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd02a6f0640 1 --2- 172.21.15.17:0/2414352258 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0240b59e0 0x7fd02414a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd02a6f0640 1 -- 172.21.15.17:0/2414352258 >> v1:172.21.15.17:6789/0 conn(0x7fd02409f5b0 legacy=0x7fd0240b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:09.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd028eed640 1 -- 172.21.15.17:0/2414352258 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1086271005 0 0) 0x7fd0240b5dc0 con 0x7fd02409f5b0 2024-03-16T10:11:09.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd02a6f0640 1 -- 172.21.15.17:0/2414352258 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02406de50 con 0x7fd024073210 2024-03-16T10:11:09.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd02a6f0640 1 --2- 172.21.15.17:0/2414352258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd024073210 0x7fd02406e370 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd01c009e50 tx=0x7fd01c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4649db501ccf1bb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:09.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd028eed640 1 -- 172.21.15.17:0/2414352258 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0a5660 con 0x7fd024073210 2024-03-16T10:11:09.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd028eed640 1 -- 172.21.15.17:0/2414352258 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd01c0a5800 con 0x7fd024073210 2024-03-16T10:11:09.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.169+0000 7fd028eed640 1 -- 172.21.15.17:0/2414352258 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd01c0a5bd0 con 0x7fd024073210 2024-03-16T10:11:09.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 -- 172.21.15.17:0/2414352258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd024073210 msgr2=0x7fd02406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:09.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 --2- 172.21.15.17:0/2414352258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd024073210 0x7fd02406e370 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd01c009e50 tx=0x7fd01c01c9a0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 -- 172.21.15.17:0/2414352258 shutdown_connections 2024-03-16T10:11:09.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 --2- 172.21.15.17:0/2414352258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd024073210 0x7fd02406e370 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 --2- 172.21.15.17:0/2414352258 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0240b59e0 0x7fd02414a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 -- 172.21.15.17:0/2414352258 >> 172.21.15.17:0/2414352258 conn(0x7fd0240b6870 msgr2=0x7fd0240b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:09.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 -- 172.21.15.17:0/2414352258 shutdown_connections 2024-03-16T10:11:09.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 -- 172.21.15.17:0/2414352258 wait complete. 2024-03-16T10:11:09.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 Processor -- start 2024-03-16T10:11:09.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 -- start start 2024-03-16T10:11:09.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd024073210 0x7fd024147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:09.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0240b59e0 0x7fd0240b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0241e37a0 0x7fd0241e01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd024154730 con 0x7fd024073210 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0240b0fb0 con 0x7fd0240b59e0 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd02c153640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd0241548b0 con 0x7fd0241e37a0 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd029eef640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd024073210 0x7fd024147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd029eef640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd024073210 0x7fd024147a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55516/0 (socket says 172.21.15.17:55516) 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd029eef640 1 -- 172.21.15.17:0/2764209480 learned_addr learned my addr 172.21.15.17:0/2764209480 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd0296ee640 1 --2- 172.21.15.17:0/2764209480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0240b59e0 0x7fd0240b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd029eef640 1 -- 172.21.15.17:0/2764209480 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0241e37a0 msgr2=0x7fd0241e01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd029eef640 1 --2- 172.21.15.17:0/2764209480 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0241e37a0 0x7fd0241e01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd029eef640 1 -- 172.21.15.17:0/2764209480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0240b59e0 msgr2=0x7fd0240b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd029eef640 1 --2- 172.21.15.17:0/2764209480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0240b59e0 0x7fd0240b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd029eef640 1 -- 172.21.15.17:0/2764209480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0241e3d10 con 0x7fd024073210 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd0296ee640 1 --2- 172.21.15.17:0/2764209480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0240b59e0 0x7fd0240b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:09.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd029eef640 1 --2- 172.21.15.17:0/2764209480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd024073210 0x7fd024147a80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd02000c820 tx=0x7fd02000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd01a7fc640 1 -- 172.21.15.17:0/2764209480 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020015070 con 0x7fd024073210 2024-03-16T10:11:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.173+0000 7fd01a7fc640 1 -- 172.21.15.17:0/2764209480 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd020007a30 con 0x7fd024073210 2024-03-16T10:11:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.177+0000 7fd02c153640 1 -- 172.21.15.17:0/2764209480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0241e3ff0 con 0x7fd024073210 2024-03-16T10:11:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.177+0000 7fd01a7fc640 1 -- 172.21.15.17:0/2764209480 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020007e20 con 0x7fd024073210 2024-03-16T10:11:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.177+0000 7fd02c153640 1 -- 172.21.15.17:0/2764209480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd024147fc0 con 0x7fd024073210 2024-03-16T10:11:09.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.177+0000 7fd02c153640 1 -- 172.21.15.17:0/2764209480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd02406a130 con 0x7fd024073210 2024-03-16T10:11:09.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.177+0000 7fd01a7fc640 1 -- 172.21.15.17:0/2764209480 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd020018070 con 0x7fd024073210 2024-03-16T10:11:09.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.177+0000 7fd01a7fc640 1 --2- 172.21.15.17:0/2764209480 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd008067660 0x7fd008069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:09.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.177+0000 7fd01a7fc640 1 -- 172.21.15.17:0/2764209480 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd020085a10 con 0x7fd024073210 2024-03-16T10:11:09.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.177+0000 7fd0296ee640 1 --2- 172.21.15.17:0/2764209480 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd008067660 0x7fd008069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:09.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.177+0000 7fd0296ee640 1 --2- 172.21.15.17:0/2764209480 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd008067660 0x7fd008069b20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd0241dd9a0 tx=0x7fd014009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:09.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.181+0000 7fd01a7fc640 1 -- 172.21.15.17:0/2764209480 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd020056800 con 0x7fd024073210 2024-03-16T10:11:09.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.345+0000 7fd02c153640 1 -- 172.21.15.17:0/2764209480 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd02406efc0 con 0x7fd024073210 2024-03-16T10:11:09.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.345+0000 7fd01a7fc640 1 -- 172.21.15.17:0/2764209480 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7fd020056100 con 0x7fd024073210 2024-03-16T10:11:09.351 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:09.351 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.349+0000 7fd02c153640 1 -- 172.21.15.17:0/2764209480 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd008067660 msgr2=0x7fd008069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.349+0000 7fd02c153640 1 --2- 172.21.15.17:0/2764209480 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd008067660 0x7fd008069b20 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd0241dd9a0 tx=0x7fd014009040 comp rx=0 tx=0).stop 2024-03-16T10:11:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.349+0000 7fd02c153640 1 -- 172.21.15.17:0/2764209480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd024073210 msgr2=0x7fd024147a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.349+0000 7fd02c153640 1 --2- 172.21.15.17:0/2764209480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd024073210 0x7fd024147a80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd02000c820 tx=0x7fd02000cce0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.349+0000 7fd02c153640 1 -- 172.21.15.17:0/2764209480 shutdown_connections 2024-03-16T10:11:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.349+0000 7fd02c153640 1 --2- 172.21.15.17:0/2764209480 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd008067660 0x7fd008069b20 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.349+0000 7fd02c153640 1 --2- 172.21.15.17:0/2764209480 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd0241e37a0 0x7fd0241e01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.349+0000 7fd02c153640 1 --2- 172.21.15.17:0/2764209480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0240b59e0 0x7fd0240b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.349+0000 7fd02c153640 1 --2- 172.21.15.17:0/2764209480 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd024073210 0x7fd024147a80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.349+0000 7fd02c153640 1 -- 172.21.15.17:0/2764209480 >> 172.21.15.17:0/2764209480 conn(0x7fd0240b6870 msgr2=0x7fd0240725d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:09.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.349+0000 7fd02c153640 1 -- 172.21.15.17:0/2764209480 shutdown_connections 2024-03-16T10:11:09.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.349+0000 7fd02c153640 1 -- 172.21.15.17:0/2764209480 wait complete. 2024-03-16T10:11:09.376 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:11:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.501+0000 7fcb9c808640 1 Processor -- start 2024-03-16T10:11:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb9c808640 1 -- start start 2024-03-16T10:11:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb9c808640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fcb98148a30 0x7fcb98148e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb9c808640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcb980b72e0 0x7fcb98149350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb9c808640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb9806f5a0 con 0x7fcb9806e660 2024-03-16T10:11:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb9c808640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fcb9806fdc0 con 0x7fcb980b72e0 2024-03-16T10:11:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb9c808640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fcb980a0e20 con 0x7fcb98148a30 2024-03-16T10:11:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb95d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fcb980b72e0 0x7fcb98149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb96d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcb9806e660 0x7fcb98142360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48924/0 (socket says 172.21.15.17:48924) 2024-03-16T10:11:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb96d9d640 1 -- 172.21.15.17:0/1937584177 learned_addr learned my addr 172.21.15.17:0/1937584177 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb9659c640 1 --2- 172.21.15.17:0/1937584177 >> v2:172.21.15.174:3300/0 conn(0x7fcb98148a30 0x7fcb98148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb95d9b640 1 -- 172.21.15.17:0/1937584177 >> v2:172.21.15.174:3300/0 conn(0x7fcb98148a30 msgr2=0x7fcb98148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb95d9b640 1 --2- 172.21.15.17:0/1937584177 >> v2:172.21.15.174:3300/0 conn(0x7fcb98148a30 0x7fcb98148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb95d9b640 1 -- 172.21.15.17:0/1937584177 >> v1:172.21.15.17:6789/0 conn(0x7fcb9806e660 legacy=0x7fcb98142360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb95d9b640 1 -- 172.21.15.17:0/1937584177 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb98142a70 con 0x7fcb980b72e0 2024-03-16T10:11:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.505+0000 7fcb9559a640 1 -- 172.21.15.17:0/1937584177 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1713980830 0 0) 0x7fcb9806f5a0 con 0x7fcb9806e660 2024-03-16T10:11:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.509+0000 7fcb95d9b640 1 --2- 172.21.15.17:0/1937584177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb980b72e0 0x7fcb98149350 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcb9806c190 tx=0x7fcb8001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7019bcbf13080771 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.509+0000 7fcb9559a640 1 -- 172.21.15.17:0/1937584177 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb800b0070 con 0x7fcb980b72e0 2024-03-16T10:11:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.509+0000 7fcb9559a640 1 -- 172.21.15.17:0/1937584177 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb800ab600 con 0x7fcb980b72e0 2024-03-16T10:11:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.509+0000 7fcb9559a640 1 -- 172.21.15.17:0/1937584177 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb800ab9d0 con 0x7fcb980b72e0 2024-03-16T10:11:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.509+0000 7fcb9c808640 1 -- 172.21.15.17:0/1937584177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb980b72e0 msgr2=0x7fcb98149350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.509+0000 7fcb9c808640 1 --2- 172.21.15.17:0/1937584177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb980b72e0 0x7fcb98149350 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcb9806c190 tx=0x7fcb8001c360 comp rx=0 tx=0).stop 2024-03-16T10:11:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.509+0000 7fcb9c808640 1 -- 172.21.15.17:0/1937584177 shutdown_connections 2024-03-16T10:11:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.509+0000 7fcb9c808640 1 --2- 172.21.15.17:0/1937584177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb980b72e0 0x7fcb98149350 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.509+0000 7fcb9c808640 1 --2- 172.21.15.17:0/1937584177 >> v2:172.21.15.174:3300/0 conn(0x7fcb98148a30 0x7fcb98148e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.509+0000 7fcb9c808640 1 -- 172.21.15.17:0/1937584177 >> 172.21.15.17:0/1937584177 conn(0x7fcb980718a0 msgr2=0x7fcb98071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.509+0000 7fcb9c808640 1 -- 172.21.15.17:0/1937584177 shutdown_connections 2024-03-16T10:11:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.509+0000 7fcb9c808640 1 -- 172.21.15.17:0/1937584177 wait complete. 2024-03-16T10:11:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.509+0000 7fcb9c808640 1 Processor -- start 2024-03-16T10:11:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.509+0000 7fcb9c808640 1 -- start start 2024-03-16T10:11:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb9c808640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb9806e660 0x7fcb98147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb9c808640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb98148a30 0x7fcb98145930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb9c808640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb98145e70 0x7fcb981441f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb9c808640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcb98152530 con 0x7fcb98145e70 2024-03-16T10:11:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb9c808640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcb980a0fa0 con 0x7fcb98148a30 2024-03-16T10:11:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb9c808640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fcb981526b0 con 0x7fcb9806e660 2024-03-16T10:11:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb96d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb98145e70 0x7fcb981441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb96d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb98145e70 0x7fcb981441f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55524/0 (socket says 172.21.15.17:55524) 2024-03-16T10:11:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb96d9d640 1 -- 172.21.15.17:0/3835470373 learned_addr learned my addr 172.21.15.17:0/3835470373 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb9659c640 1 --2- 172.21.15.17:0/3835470373 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb9806e660 0x7fcb98147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb95d9b640 1 --2- 172.21.15.17:0/3835470373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb98148a30 0x7fcb98145930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:09.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb96d9d640 1 -- 172.21.15.17:0/3835470373 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb9806e660 msgr2=0x7fcb98147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:09.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb96d9d640 1 --2- 172.21.15.17:0/3835470373 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb9806e660 0x7fcb98147880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb96d9d640 1 -- 172.21.15.17:0/3835470373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb98148a30 msgr2=0x7fcb98145930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:09.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb96d9d640 1 --2- 172.21.15.17:0/3835470373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb98148a30 0x7fcb98145930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb96d9d640 1 -- 172.21.15.17:0/3835470373 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb9813f790 con 0x7fcb98145e70 2024-03-16T10:11:09.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb96d9d640 1 --2- 172.21.15.17:0/3835470373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb98145e70 0x7fcb981441f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800c450 tx=0x7fcb8800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:09.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb8effd640 1 -- 172.21.15.17:0/3835470373 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb880074e0 con 0x7fcb98145e70 2024-03-16T10:11:09.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb8effd640 1 -- 172.21.15.17:0/3835470373 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb880047f0 con 0x7fcb98145e70 2024-03-16T10:11:09.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb9c808640 1 -- 172.21.15.17:0/3835470373 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb98144c50 con 0x7fcb98145e70 2024-03-16T10:11:09.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb8effd640 1 -- 172.21.15.17:0/3835470373 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88007680 con 0x7fcb98145e70 2024-03-16T10:11:09.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb9c808640 1 -- 172.21.15.17:0/3835470373 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb98143510 con 0x7fcb98145e70 2024-03-16T10:11:09.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb8effd640 1 -- 172.21.15.17:0/3835470373 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fcb88011070 con 0x7fcb98145e70 2024-03-16T10:11:09.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb9c808640 1 -- 172.21.15.17:0/3835470373 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb98147ed0 con 0x7fcb98145e70 2024-03-16T10:11:09.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb8effd640 1 --2- 172.21.15.17:0/3835470373 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fcb78067660 0x7fcb78069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:09.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.513+0000 7fcb8effd640 1 -- 172.21.15.17:0/3835470373 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcb88085420 con 0x7fcb98145e70 2024-03-16T10:11:09.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.517+0000 7fcb9659c640 1 --2- 172.21.15.17:0/3835470373 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fcb78067660 0x7fcb78069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:09.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.517+0000 7fcb9659c640 1 --2- 172.21.15.17:0/3835470373 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fcb78067660 0x7fcb78069b20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcb98140650 tx=0x7fcb9000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:09.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.525+0000 7fcb8effd640 1 -- 172.21.15.17:0/3835470373 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcb88056110 con 0x7fcb98145e70 2024-03-16T10:11:09.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.877+0000 7fcb9c808640 1 -- 172.21.15.17:0/3835470373 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb98141090 con 0x7fcb98145e70 2024-03-16T10:11:09.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.877+0000 7fcb8effd640 1 -- 172.21.15.17:0/3835470373 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7fcb88055a10 con 0x7fcb98145e70 2024-03-16T10:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:09.881 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1058712158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1058712158},{"type":"v1","addr":"172.21.15.174:6835","nonce":1058712158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:10:57.464714+0000","modified":"2024-03-16T10:11:00.509595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6833/3105182870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3105182870},{"type":"v1","addr":"172.21.15.17:6833","nonce":3105182870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-16T10:11:09.882 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 8 2024-03-16T10:11:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.881+0000 7fcb9c808640 1 -- 172.21.15.17:0/3835470373 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fcb78067660 msgr2=0x7fcb78069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.881+0000 7fcb9c808640 1 --2- 172.21.15.17:0/3835470373 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fcb78067660 0x7fcb78069b20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcb98140650 tx=0x7fcb9000b040 comp rx=0 tx=0).stop 2024-03-16T10:11:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.881+0000 7fcb9c808640 1 -- 172.21.15.17:0/3835470373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb98145e70 msgr2=0x7fcb981441f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.881+0000 7fcb9c808640 1 --2- 172.21.15.17:0/3835470373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb98145e70 0x7fcb981441f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800c450 tx=0x7fcb8800c910 comp rx=0 tx=0).stop 2024-03-16T10:11:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.881+0000 7fcb9c808640 1 -- 172.21.15.17:0/3835470373 shutdown_connections 2024-03-16T10:11:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.881+0000 7fcb9c808640 1 --2- 172.21.15.17:0/3835470373 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fcb78067660 0x7fcb78069b20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.881+0000 7fcb9c808640 1 --2- 172.21.15.17:0/3835470373 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcb98145e70 0x7fcb981441f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.881+0000 7fcb9c808640 1 --2- 172.21.15.17:0/3835470373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb98148a30 0x7fcb98145930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.881+0000 7fcb9c808640 1 --2- 172.21.15.17:0/3835470373 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcb9806e660 0x7fcb98147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.881+0000 7fcb9c808640 1 -- 172.21.15.17:0/3835470373 >> 172.21.15.17:0/3835470373 conn(0x7fcb980718a0 msgr2=0x7fcb980b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.881+0000 7fcb9c808640 1 -- 172.21.15.17:0/3835470373 shutdown_connections 2024-03-16T10:11:09.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:09.881+0000 7fcb9c808640 1 -- 172.21.15.17:0/3835470373 wait complete. 2024-03-16T10:11:10.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-16T10:10:57.464714+0000', 'modified': '2024-03-16T10:11:00.509595+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4263}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4263': {'gid': 4263, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6833/3105182870', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 3105182870}, {'type': 'v1', 'addr': '172.21.15.17:6833', 'nonce': 3105182870}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4263, 'qdb_cluster': [4263]} 2024-03-16T10:11:10.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-16T10:11:10.102 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-16T10:11:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f4435f3d640 1 Processor -- start 2024-03-16T10:11:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f4435f3d640 1 -- start start 2024-03-16T10:11:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f4435f3d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f44300b5110 0x7f44301445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f4435f3d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44300af0d0 0x7f443013d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f4435f3d640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44300af510 con 0x7f443013dd70 2024-03-16T10:11:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f4435f3d640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f443013e1c0 con 0x7f44300b5110 2024-03-16T10:11:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f4435f3d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f44300b54f0 con 0x7f44300af0d0 2024-03-16T10:11:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f442f7fe640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f44300b5110 0x7f44301445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f4427fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44300af0d0 0x7f443013d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f442ffff640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f443013dd70 0x7f44300b4240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.17:33582/0 (socket says 172.21.15.17:33582) 2024-03-16T10:11:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f442ffff640 1 -- 172.21.15.17:0/499579903 learned_addr learned my addr 172.21.15.17:0/499579903 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f442effd640 1 -- 172.21.15.17:0/499579903 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2267688504 0 0) 0x7f44300af510 con 0x7f443013dd70 2024-03-16T10:11:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f442effd640 1 -- 172.21.15.17:0/499579903 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f441c003530 con 0x7f443013dd70 2024-03-16T10:11:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f4427fff640 1 -- 172.21.15.17:0/499579903 >> v2:172.21.15.174:3300/0 conn(0x7f44300b5110 msgr2=0x7f44301445c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f4427fff640 1 --2- 172.21.15.17:0/499579903 >> v2:172.21.15.174:3300/0 conn(0x7f44300b5110 0x7f44301445c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f4427fff640 1 -- 172.21.15.17:0/499579903 >> v1:172.21.15.174:6789/0 conn(0x7f443013dd70 legacy=0x7f44300b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f4427fff640 1 -- 172.21.15.17:0/499579903 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4430143ae0 con 0x7f44300af0d0 2024-03-16T10:11:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.221+0000 7f442f7fe640 1 --2- 172.21.15.17:0/499579903 >> v2:172.21.15.174:3300/0 conn(0x7f44300b5110 0x7f44301445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:11:10.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.225+0000 7f4427fff640 1 --2- 172.21.15.17:0/499579903 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44300af0d0 0x7f443013d830 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f441800a960 tx=0x7f441801d840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53fbff1b3a642d32 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.225+0000 7f442effd640 1 -- 172.21.15.17:0/499579903 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44180b3070 con 0x7f44300af0d0 2024-03-16T10:11:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.225+0000 7f442effd640 1 -- 172.21.15.17:0/499579903 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44180a67a0 con 0x7f44300af0d0 2024-03-16T10:11:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.225+0000 7f442effd640 1 -- 172.21.15.17:0/499579903 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441800b040 con 0x7f44300af0d0 2024-03-16T10:11:10.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.225+0000 7f4435f3d640 1 -- 172.21.15.17:0/499579903 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44300af0d0 msgr2=0x7f443013d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.225+0000 7f4435f3d640 1 --2- 172.21.15.17:0/499579903 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44300af0d0 0x7f443013d830 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f441800a960 tx=0x7f441801d840 comp rx=0 tx=0).stop 2024-03-16T10:11:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.225+0000 7f4435f3d640 1 -- 172.21.15.17:0/499579903 shutdown_connections 2024-03-16T10:11:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.225+0000 7f4435f3d640 1 --2- 172.21.15.17:0/499579903 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44300af0d0 0x7f443013d830 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.225+0000 7f4435f3d640 1 --2- 172.21.15.17:0/499579903 >> v2:172.21.15.174:3300/0 conn(0x7f44300b5110 0x7f44301445c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.225+0000 7f4435f3d640 1 -- 172.21.15.17:0/499579903 >> 172.21.15.17:0/499579903 conn(0x7f443009b130 msgr2=0x7f443009b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.225+0000 7f4435f3d640 1 -- 172.21.15.17:0/499579903 shutdown_connections 2024-03-16T10:11:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.225+0000 7f4435f3d640 1 -- 172.21.15.17:0/499579903 wait complete. 2024-03-16T10:11:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.225+0000 7f4435f3d640 1 Processor -- start 2024-03-16T10:11:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.225+0000 7f4435f3d640 1 -- start start 2024-03-16T10:11:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f4435f3d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44300af0d0 0x7f443009e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f4435f3d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44300b5110 0x7f44301e0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f4435f3d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f443013dd70 0x7f44301e0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f4435f3d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f44301506d0 con 0x7f44300af0d0 2024-03-16T10:11:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f4435f3d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4430150550 con 0x7f443013dd70 2024-03-16T10:11:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f4435f3d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4430150850 con 0x7f44300b5110 2024-03-16T10:11:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44300af0d0 0x7f443009e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44300af0d0 0x7f443009e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55528/0 (socket says 172.21.15.17:55528) 2024-03-16T10:11:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442f7fe640 1 -- 172.21.15.17:0/2699000190 learned_addr learned my addr 172.21.15.17:0/2699000190 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442f7fe640 1 -- 172.21.15.17:0/2699000190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44300b5110 msgr2=0x7f44301e0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442f7fe640 1 --2- 172.21.15.17:0/2699000190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44300b5110 0x7f44301e0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442f7fe640 1 -- 172.21.15.17:0/2699000190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f443013dd70 msgr2=0x7f44301e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442f7fe640 1 --2- 172.21.15.17:0/2699000190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f443013dd70 0x7f44301e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442f7fe640 1 -- 172.21.15.17:0/2699000190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4430142f40 con 0x7f44300af0d0 2024-03-16T10:11:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442f7fe640 1 --2- 172.21.15.17:0/2699000190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44300af0d0 0x7f443009e7b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f442000ebe0 tx=0x7f442000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442cff9640 1 -- 172.21.15.17:0/2699000190 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420017070 con 0x7f44300af0d0 2024-03-16T10:11:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442cff9640 1 -- 172.21.15.17:0/2699000190 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f442000cc30 con 0x7f44300af0d0 2024-03-16T10:11:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f4435f3d640 1 -- 172.21.15.17:0/2699000190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4430070e60 con 0x7f44300af0d0 2024-03-16T10:11:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442cff9640 1 -- 172.21.15.17:0/2699000190 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420013470 con 0x7f44300af0d0 2024-03-16T10:11:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f4435f3d640 1 -- 172.21.15.17:0/2699000190 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f443009d490 con 0x7f44300af0d0 2024-03-16T10:11:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442cff9640 1 -- 172.21.15.17:0/2699000190 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f442001a070 con 0x7f44300af0d0 2024-03-16T10:11:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442cff9640 1 --2- 172.21.15.17:0/2699000190 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4410067660 0x7f4410069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f442cff9640 1 -- 172.21.15.17:0/2699000190 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f44200857a0 con 0x7f44300af0d0 2024-03-16T10:11:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f4427fff640 1 --2- 172.21.15.17:0/2699000190 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4410067660 0x7f4410069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.229+0000 7f4427fff640 1 --2- 172.21.15.17:0/2699000190 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4410067660 0x7f4410069b20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f44180b0000 tx=0x7f4418004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 Processor -- start 2024-03-16T10:11:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 -- start start 2024-03-16T10:11:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc910e44a0 0x55bc910e4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e4dc0 0x55bc9110f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc9110f6c0 0x55bc9110faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x55bc90f2d540 con 0x55bc9110f6c0 2024-03-16T10:11:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc90eb4b00 con 0x55bc910e4dc0 2024-03-16T10:11:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55bc90f295e0 con 0x55bc910e44a0 2024-03-16T10:11:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f442ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc9110f6c0 0x55bc9110faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f442ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc9110f6c0 0x55bc9110faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55532/0 (socket says 172.21.15.17:55532) 2024-03-16T10:11:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f442ffff640 1 -- 172.21.15.17:0/1012559025 learned_addr learned my addr 172.21.15.17:0/1012559025 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f442ffff640 1 -- 172.21.15.17:0/1012559025 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc910e44a0 msgr2=0x55bc910e4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f442ffff640 1 --2- 172.21.15.17:0/1012559025 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc910e44a0 0x55bc910e4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f442ffff640 1 -- 172.21.15.17:0/1012559025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e4dc0 msgr2=0x55bc9110f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f4427fff640 1 --2- 172.21.15.17:0/1012559025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e4dc0 0x55bc9110f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f442ffff640 1 --2- 172.21.15.17:0/1012559025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e4dc0 0x55bc9110f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f442ffff640 1 -- 172.21.15.17:0/1012559025 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc90eea6a0 con 0x55bc9110f6c0 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f442ffff640 1 --2- 172.21.15.17:0/1012559025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc9110f6c0 0x55bc9110faa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f44280020e0 tx=0x7f442800c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4ce5a5d1beb3b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44257fa640 1 -- 172.21.15.17:0/1012559025 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428017070 con 0x55bc9110f6c0 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44257fa640 1 -- 172.21.15.17:0/1012559025 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44280072d0 con 0x55bc9110f6c0 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44257fa640 1 -- 172.21.15.17:0/1012559025 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44280076c0 con 0x55bc9110f6c0 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 -- 172.21.15.17:0/1012559025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc9110f6c0 msgr2=0x55bc9110faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 --2- 172.21.15.17:0/1012559025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc9110f6c0 0x55bc9110faa0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f44280020e0 tx=0x7f442800c4b0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 -- 172.21.15.17:0/1012559025 shutdown_connections 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 --2- 172.21.15.17:0/1012559025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc9110f6c0 0x55bc9110faa0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 --2- 172.21.15.17:0/1012559025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e4dc0 0x55bc9110f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 --2- 172.21.15.17:0/1012559025 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc910e44a0 0x55bc910e4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 -- 172.21.15.17:0/1012559025 >> 172.21.15.17:0/1012559025 conn(0x55bc9110af30 msgr2=0x55bc9110d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 -- 172.21.15.17:0/1012559025 shutdown_connections 2024-03-16T10:11:10.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.233+0000 7f44390bf000 1 -- 172.21.15.17:0/1012559025 wait complete. 2024-03-16T10:11:10.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 -- start start 2024-03-16T10:11:10.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 Processor -- start 2024-03-16T10:11:10.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e44a0 0x55bc910dde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e4dc0 0x55bc910de390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc9110f6c0 0x55bc910de8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x55bc90f31b70 con 0x55bc910e44a0 2024-03-16T10:11:10.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc90eeb850 con 0x55bc910e4dc0 2024-03-16T10:11:10.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55bc91111020 con 0x55bc9110f6c0 2024-03-16T10:11:10.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 10 client.0 _ll_register_callbacks cb 0x55bc910ca220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:11:10.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f442f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e44a0 0x55bc910dde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f442f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e44a0 0x55bc910dde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55548/0 (socket says 172.21.15.17:55548) 2024-03-16T10:11:10.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f442f7fe640 1 -- 172.21.15.17:0/3099616840 learned_addr learned my addr 172.21.15.17:0/3099616840 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:10.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f442f7fe640 1 -- 172.21.15.17:0/3099616840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc9110f6c0 msgr2=0x55bc910de8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f442f7fe640 1 --2- 172.21.15.17:0/3099616840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc9110f6c0 0x55bc910de8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f442f7fe640 1 -- 172.21.15.17:0/3099616840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e4dc0 msgr2=0x55bc910de390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f442f7fe640 1 --2- 172.21.15.17:0/3099616840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e4dc0 0x55bc910de390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f442f7fe640 1 -- 172.21.15.17:0/3099616840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc90ee8900 con 0x55bc910e44a0 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f442f7fe640 1 --2- 172.21.15.17:0/3099616840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e44a0 0x55bc910dde50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4420055d40 tx=0x7f4420056570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7d3c85fe2cd007 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f440e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f440e7fc640 1 -- 172.21.15.17:0/3099616840 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420021070 con 0x55bc910e44a0 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f440e7fc640 1 -- 172.21.15.17:0/3099616840 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4420055200 con 0x55bc910e44a0 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 -- 172.21.15.17:0/3099616840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44280072d0 con 0x55bc910e44a0 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f440e7fc640 1 -- 172.21.15.17:0/3099616840 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f442005e4c0 con 0x55bc910e44a0 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 10 client.4614 fetch_fsmap learned FSMap version 8 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 -- 172.21.15.17:0/3099616840 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55bc90fae940 con 0x55bc910e44a0 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f440e7fc640 1 -- 172.21.15.17:0/3099616840 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44280072d0 con 0x55bc910e44a0 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f440e7fc640 1 -- 172.21.15.17:0/3099616840 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 8) ==== 2284+0+0 (secure 0 0 0) 0x7f4420024060 con 0x55bc910e44a0 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 10 client.4614 fetch_fsmap finished waiting for FSMap version 8 2024-03-16T10:11:10.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 10 client.4614 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-16T10:11:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 20 client.4614 populate_metadata read hostname 'smithi017' 2024-03-16T10:11:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 --2- 172.21.15.17:0/3099616840 >> [v2:172.21.15.17:6832/3105182870,v1:172.21.15.17:6833/3105182870] conn(0x55bc91122560 0x55bc91124950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 4 client.4614 mds_command: new command op to 4263 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T10:11:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 -- 172.21.15.17:0/3099616840 --> [v2:172.21.15.17:6832/3105182870,v1:172.21.15.17:6833/3105182870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc90ec8bc0 con 0x55bc91122560 2024-03-16T10:11:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f4427fff640 1 --2- 172.21.15.17:0/3099616840 >> [v2:172.21.15.17:6832/3105182870,v1:172.21.15.17:6833/3105182870] conn(0x55bc91122560 0x55bc91124950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f4427fff640 1 --2- 172.21.15.17:0/3099616840 >> [v2:172.21.15.17:6832/3105182870,v1:172.21.15.17:6833/3105182870] conn(0x55bc91122560 0x55bc91124950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d8f1504631f196d server_cookie=56d9d0fd90e98176 in_seq=0 out_seq=0 2024-03-16T10:11:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f440e7fc640 10 client.4614 ms_handle_connect on v2:172.21.15.17:6832/3105182870 2024-03-16T10:11:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f440e7fc640 1 -- 172.21.15.17:0/3099616840 <== mds.0 v2:172.21.15.17:6832/3105182870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55bc90ec8bc0 con 0x55bc91122560 2024-03-16T10:11:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f440e7fc640 10 client.4614 handle_command_reply: tid=0 2024-03-16T10:11:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 client.4614 shutdown 2024-03-16T10:11:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 -- 172.21.15.17:0/3099616840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e44a0 msgr2=0x55bc910dde50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 --2- 172.21.15.17:0/3099616840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e44a0 0x55bc910dde50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4420055d40 tx=0x7f4420056570 comp rx=0 tx=0).stop 2024-03-16T10:11:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 -- 172.21.15.17:0/3099616840 shutdown_connections 2024-03-16T10:11:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 --2- 172.21.15.17:0/3099616840 >> [v2:172.21.15.17:6832/3105182870,v1:172.21.15.17:6833/3105182870] conn(0x55bc91122560 0x55bc91124950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 --2- 172.21.15.17:0/3099616840 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc9110f6c0 0x55bc910de8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 --2- 172.21.15.17:0/3099616840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e4dc0 0x55bc910de390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f440e7fc640 0 client.4614 ms_handle_reset on v2:172.21.15.17:6832/3105182870 2024-03-16T10:11:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 --2- 172.21.15.17:0/3099616840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e44a0 0x55bc910dde50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 -- 172.21.15.17:0/3099616840 >> 172.21.15.17:0/3099616840 conn(0x55bc9110af30 msgr2=0x55bc9110d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 -- 172.21.15.17:0/3099616840 shutdown_connections 2024-03-16T10:11:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 1 -- 172.21.15.17:0/3099616840 wait complete. 2024-03-16T10:11:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.237+0000 7f44390bf000 20 client.4614 trim_cache size 0 max 16384 2024-03-16T10:11:10.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 Processor -- start 2024-03-16T10:11:10.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 -- start start 2024-03-16T10:11:10.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e44a0 0x55bc9110e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e4dc0 0x55bc9117c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc9110f6c0 0x55bc9117cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f44301ec7c0 con 0x55bc910e4dc0 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44301eb900 con 0x55bc910e44a0 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55bc90fae4b0 con 0x55bc9110f6c0 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f4427fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e4dc0 0x55bc9117c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f4427fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e4dc0 0x55bc9117c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55562/0 (socket says 172.21.15.17:55562) 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f4427fff640 1 -- 172.21.15.17:0/1675280991 learned_addr learned my addr 172.21.15.17:0/1675280991 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f4427fff640 1 -- 172.21.15.17:0/1675280991 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc9110f6c0 msgr2=0x55bc9117cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f4427fff640 1 --2- 172.21.15.17:0/1675280991 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc9110f6c0 0x55bc9117cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f4427fff640 1 -- 172.21.15.17:0/1675280991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e44a0 msgr2=0x55bc9110e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f4427fff640 1 --2- 172.21.15.17:0/1675280991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e44a0 0x55bc9110e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f4427fff640 1 -- 172.21.15.17:0/1675280991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc90f85870 con 0x55bc910e4dc0 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f4427fff640 1 --2- 172.21.15.17:0/1675280991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e4dc0 0x55bc9117c7e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f44180b1040 tx=0x7f44180a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d78e0925fdf559 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f4424ff9640 1 -- 172.21.15.17:0/1675280991 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44180b3040 con 0x55bc910e4dc0 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f4424ff9640 1 -- 172.21.15.17:0/1675280991 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44180c8160 con 0x55bc910e4dc0 2024-03-16T10:11:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f4424ff9640 1 -- 172.21.15.17:0/1675280991 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44180c8510 con 0x55bc910e4dc0 2024-03-16T10:11:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 -- 172.21.15.17:0/1675280991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e4dc0 msgr2=0x55bc9117c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 --2- 172.21.15.17:0/1675280991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e4dc0 0x55bc9117c7e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f44180b1040 tx=0x7f44180a8000 comp rx=0 tx=0).stop 2024-03-16T10:11:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 -- 172.21.15.17:0/1675280991 shutdown_connections 2024-03-16T10:11:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 --2- 172.21.15.17:0/1675280991 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc9110f6c0 0x55bc9117cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 --2- 172.21.15.17:0/1675280991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e4dc0 0x55bc9117c7e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 --2- 172.21.15.17:0/1675280991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e44a0 0x55bc9110e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 -- 172.21.15.17:0/1675280991 >> 172.21.15.17:0/1675280991 conn(0x55bc9110af30 msgr2=0x55bc91129fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 -- 172.21.15.17:0/1675280991 shutdown_connections 2024-03-16T10:11:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 -- 172.21.15.17:0/1675280991 wait complete. 2024-03-16T10:11:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.257+0000 7f44390bf000 1 -- start start 2024-03-16T10:11:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 Processor -- start 2024-03-16T10:11:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e44a0 0x55bc9117c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e4dc0 0x55bc9117cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc9110f6c0 0x55bc91128130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x55bc90ec57b0 con 0x55bc910e44a0 2024-03-16T10:11:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc90ee7f50 con 0x55bc910e4dc0 2024-03-16T10:11:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55bc90f2c850 con 0x55bc9110f6c0 2024-03-16T10:11:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 10 client.0 _ll_register_callbacks cb 0x7f442000eee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:11:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f442f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e44a0 0x55bc9117c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f442f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e44a0 0x55bc9117c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55572/0 (socket says 172.21.15.17:55572) 2024-03-16T10:11:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f442f7fe640 1 -- 172.21.15.17:0/1465726973 learned_addr learned my addr 172.21.15.17:0/1465726973 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f442ffff640 1 --2- 172.21.15.17:0/1465726973 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc9110f6c0 0x55bc91128130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4427fff640 1 --2- 172.21.15.17:0/1465726973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e4dc0 0x55bc9117cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f442f7fe640 1 -- 172.21.15.17:0/1465726973 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc9110f6c0 msgr2=0x55bc91128130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f442f7fe640 1 --2- 172.21.15.17:0/1465726973 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc9110f6c0 0x55bc91128130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f442f7fe640 1 -- 172.21.15.17:0/1465726973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e4dc0 msgr2=0x55bc9117cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f442f7fe640 1 --2- 172.21.15.17:0/1465726973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e4dc0 0x55bc9117cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f442f7fe640 1 -- 172.21.15.17:0/1465726973 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc90edd070 con 0x55bc910e44a0 2024-03-16T10:11:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f442f7fe640 1 --2- 172.21.15.17:0/1465726973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e44a0 0x55bc9117c9c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f442005df90 tx=0x7f4420000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=135ac68c8bd79f9e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f440e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-16T10:11:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f440e7fc640 1 -- 172.21.15.17:0/1465726973 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4420051b00 con 0x55bc910e44a0 2024-03-16T10:11:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f440e7fc640 1 -- 172.21.15.17:0/1465726973 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4420051ca0 con 0x55bc910e44a0 2024-03-16T10:11:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 -- 172.21.15.17:0/1465726973 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44180c8160 con 0x55bc910e44a0 2024-03-16T10:11:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f440e7fc640 1 -- 172.21.15.17:0/1465726973 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f4420054c70 con 0x55bc910e44a0 2024-03-16T10:11:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 10 client.4620 fetch_fsmap learned FSMap version 8 2024-03-16T10:11:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 -- 172.21.15.17:0/1465726973 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55bc90ec1370 con 0x55bc910e44a0 2024-03-16T10:11:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f440e7fc640 1 -- 172.21.15.17:0/1465726973 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44180c8160 con 0x55bc910e44a0 2024-03-16T10:11:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f440e7fc640 1 -- 172.21.15.17:0/1465726973 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 8) ==== 2284+0+0 (secure 0 0 0) 0x7f4420054e10 con 0x55bc910e44a0 2024-03-16T10:11:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 10 client.4620 fetch_fsmap finished waiting for FSMap version 8 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 10 client.4620 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 20 client.4620 populate_metadata read hostname 'smithi017' 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 --2- 172.21.15.17:0/1465726973 >> [v2:172.21.15.17:6832/3105182870,v1:172.21.15.17:6833/3105182870] conn(0x55bc91128af0 0x55bc91128ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 4 client.4620 mds_command: new command op to 4263 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 -- 172.21.15.17:0/1465726973 --> [v2:172.21.15.17:6832/3105182870,v1:172.21.15.17:6833/3105182870] -- command(tid 0: {"prefix": "status"}) -- 0x55bc90f3aa80 con 0x55bc91128af0 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4427fff640 1 --2- 172.21.15.17:0/1465726973 >> [v2:172.21.15.17:6832/3105182870,v1:172.21.15.17:6833/3105182870] conn(0x55bc91128af0 0x55bc91128ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4427fff640 1 --2- 172.21.15.17:0/1465726973 >> [v2:172.21.15.17:6832/3105182870,v1:172.21.15.17:6833/3105182870] conn(0x55bc91128af0 0x55bc91128ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb60120573016731 server_cookie=6f525000ccbd5db9 in_seq=0 out_seq=0 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f440e7fc640 10 client.4620 ms_handle_connect on v2:172.21.15.17:6832/3105182870 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f440e7fc640 1 -- 172.21.15.17:0/1465726973 <== mds.0 v2:172.21.15.17:6832/3105182870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55bc90f3aa80 con 0x55bc91128af0 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f440e7fc640 10 client.4620 handle_command_reply: tid=0 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 client.4620 shutdown 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 -- 172.21.15.17:0/1465726973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e44a0 msgr2=0x55bc9117c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 --2- 172.21.15.17:0/1465726973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e44a0 0x55bc9117c9c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f442005df90 tx=0x7f4420000bd0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 -- 172.21.15.17:0/1465726973 shutdown_connections 2024-03-16T10:11:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 --2- 172.21.15.17:0/1465726973 >> [v2:172.21.15.17:6832/3105182870,v1:172.21.15.17:6833/3105182870] conn(0x55bc91128af0 0x55bc91128ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 --2- 172.21.15.17:0/1465726973 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55bc9110f6c0 0x55bc91128130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 --2- 172.21.15.17:0/1465726973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc910e4dc0 0x55bc9117cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f440e7fc640 0 client.4620 ms_handle_reset on v2:172.21.15.17:6832/3105182870 2024-03-16T10:11:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 --2- 172.21.15.17:0/1465726973 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55bc910e44a0 0x55bc9117c9c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 -- 172.21.15.17:0/1465726973 >> 172.21.15.17:0/1465726973 conn(0x55bc9110af30 msgr2=0x55bc9110d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 -- 172.21.15.17:0/1465726973 shutdown_connections 2024-03-16T10:11:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 1 -- 172.21.15.17:0/1465726973 wait complete. 2024-03-16T10:11:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f44390bf000 20 client.4620 trim_cache size 0 max 16384 2024-03-16T10:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-16T10:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "b21729d9-93de-4054-b789-790eefecfc1c", 2024-03-16T10:11:10.271 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-16T10:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4263, 2024-03-16T10:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-16T10:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-16T10:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-16T10:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 12.122790613999999, 2024-03-16T10:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 8, 2024-03-16T10:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 21, 2024-03-16T10:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 21, 2024-03-16T10:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 19.080328225999999 2024-03-16T10:11:10.272 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-16T10:11:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4435f3d640 1 -- 172.21.15.17:0/2699000190 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4410067660 msgr2=0x7f4410069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4435f3d640 1 --2- 172.21.15.17:0/2699000190 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4410067660 0x7f4410069b20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f44180b0000 tx=0x7f4418004740 comp rx=0 tx=0).stop 2024-03-16T10:11:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4435f3d640 1 -- 172.21.15.17:0/2699000190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44300af0d0 msgr2=0x7f443009e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4435f3d640 1 --2- 172.21.15.17:0/2699000190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44300af0d0 0x7f443009e7b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f442000ebe0 tx=0x7f442000c520 comp rx=0 tx=0).stop 2024-03-16T10:11:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4435f3d640 1 -- 172.21.15.17:0/2699000190 shutdown_connections 2024-03-16T10:11:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4435f3d640 1 --2- 172.21.15.17:0/2699000190 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4410067660 0x7f4410069b20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4435f3d640 1 --2- 172.21.15.17:0/2699000190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f443013dd70 0x7f44301e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4435f3d640 1 --2- 172.21.15.17:0/2699000190 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44300b5110 0x7f44301e0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4435f3d640 1 --2- 172.21.15.17:0/2699000190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44300af0d0 0x7f443009e7b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4435f3d640 1 -- 172.21.15.17:0/2699000190 >> 172.21.15.17:0/2699000190 conn(0x7f443009b130 msgr2=0x7f4430142830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4435f3d640 1 -- 172.21.15.17:0/2699000190 shutdown_connections 2024-03-16T10:11:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.261+0000 7f4435f3d640 1 -- 172.21.15.17:0/2699000190 wait complete. 2024-03-16T10:11:10.282 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-16T10:11:10.304 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-16T10:11:10.304 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-16T10:11:10.304 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-16T10:11:10.305 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-16T10:11:10.305 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:10.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:10.305 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-03-16T10:11:10.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:10.311 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link delete ceph-brx 2024-03-16T10:11:10.364 INFO:teuthology.orchestra.run.smithi017.stderr:Cannot find device "ceph-brx" 2024-03-16T10:11:10.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:11:10.366 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-16T10:11:10.366 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-16T10:11:10.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.517+0000 7f5bfcd7a640 1 Processor -- start 2024-03-16T10:11:10.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.517+0000 7f5bfcd7a640 1 -- start start 2024-03-16T10:11:10.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.517+0000 7f5bfcd7a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5bf80b5710 0x7f5bf80b0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.517+0000 7f5bfcd7a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bf806de30 0x7f5bf8151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.517+0000 7f5bfcd7a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bf809f600 con 0x7f5bf806e720 2024-03-16T10:11:10.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.517+0000 7f5bfcd7a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f5bf809fde0 con 0x7f5bf80b5710 2024-03-16T10:11:10.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.517+0000 7f5bfcd7a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf8073390 con 0x7f5bf806de30 2024-03-16T10:11:10.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bf659c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5bf80b5710 0x7f5bf80b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bf6d9d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bf806de30 0x7f5bf8151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bedd9b640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f5bf806e720 0x7f5bf8147460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.17:33596/0 (socket says 172.21.15.17:33596) 2024-03-16T10:11:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bedd9b640 1 -- 172.21.15.17:0/2709335395 learned_addr learned my addr 172.21.15.17:0/2709335395 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bf659c640 1 -- 172.21.15.17:0/2709335395 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bf806de30 msgr2=0x7f5bf8151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bf659c640 1 --2- 172.21.15.17:0/2709335395 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bf806de30 0x7f5bf8151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bf659c640 1 -- 172.21.15.17:0/2709335395 >> v1:172.21.15.174:6789/0 conn(0x7f5bf806e720 legacy=0x7f5bf8147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bf659c640 1 -- 172.21.15.17:0/2709335395 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf809f8c0 con 0x7f5bf80b5710 2024-03-16T10:11:10.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bf659c640 1 --2- 172.21.15.17:0/2709335395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bf80b5710 0x7f5bf80b0790 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5be80097c0 tx=0x7f5be801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4175eba843e9dec server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:10.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bf5d9b640 1 -- 172.21.15.17:0/2709335395 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be80b0070 con 0x7f5bf80b5710 2024-03-16T10:11:10.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bf5d9b640 1 -- 172.21.15.17:0/2709335395 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be80ab600 con 0x7f5bf80b5710 2024-03-16T10:11:10.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bf5d9b640 1 -- 172.21.15.17:0/2709335395 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be80ab9d0 con 0x7f5bf80b5710 2024-03-16T10:11:10.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/2709335395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bf80b5710 msgr2=0x7f5bf80b0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bfcd7a640 1 --2- 172.21.15.17:0/2709335395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bf80b5710 0x7f5bf80b0790 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5be80097c0 tx=0x7f5be801c410 comp rx=0 tx=0).stop 2024-03-16T10:11:10.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/2709335395 shutdown_connections 2024-03-16T10:11:10.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bfcd7a640 1 --2- 172.21.15.17:0/2709335395 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bf806de30 0x7f5bf8151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bfcd7a640 1 --2- 172.21.15.17:0/2709335395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bf80b5710 0x7f5bf80b0790 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/2709335395 >> 172.21.15.17:0/2709335395 conn(0x7f5bf80b6800 msgr2=0x7f5bf80b6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:10.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/2709335395 shutdown_connections 2024-03-16T10:11:10.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.521+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/2709335395 wait complete. 2024-03-16T10:11:10.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bfcd7a640 1 Processor -- start 2024-03-16T10:11:10.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bfcd7a640 1 -- start start 2024-03-16T10:11:10.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bfcd7a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bf80b5710 0x7f5bf813d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bfcd7a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bf80b3cf0 0x7f5bf8144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bfcd7a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bf8142ed0 0x7f5bf81e4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bfcd7a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf8154eb0 con 0x7f5bf8142ed0 2024-03-16T10:11:10.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bfcd7a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf8154d30 con 0x7f5bf80b5710 2024-03-16T10:11:10.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bfcd7a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf8155030 con 0x7f5bf80b3cf0 2024-03-16T10:11:10.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bf6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bf8142ed0 0x7f5bf81e4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bf6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bf8142ed0 0x7f5bf81e4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55576/0 (socket says 172.21.15.17:55576) 2024-03-16T10:11:10.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bf6d9d640 1 -- 172.21.15.17:0/3831422993 learned_addr learned my addr 172.21.15.17:0/3831422993 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:10.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bf659c640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bf80b5710 0x7f5bf813d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bedd9b640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bf80b3cf0 0x7f5bf8144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bf6d9d640 1 -- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bf80b3cf0 msgr2=0x7f5bf8144640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bf6d9d640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bf80b3cf0 0x7f5bf8144640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bf6d9d640 1 -- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bf80b5710 msgr2=0x7f5bf813d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bf6d9d640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bf80b5710 0x7f5bf813d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bf6d9d640 1 -- 172.21.15.17:0/3831422993 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf8143360 con 0x7f5bf8142ed0 2024-03-16T10:11:10.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bf659c640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bf80b5710 0x7f5bf813d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:10.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bf6d9d640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bf8142ed0 0x7f5bf81e4260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009d30 tx=0x7f5bf000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:10.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bef7fe640 1 -- 172.21.15.17:0/3831422993 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0018020 con 0x7f5bf8142ed0 2024-03-16T10:11:10.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bef7fe640 1 -- 172.21.15.17:0/3831422993 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf0014480 con 0x7f5bf8142ed0 2024-03-16T10:11:10.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/3831422993 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf80b1b80 con 0x7f5bf8142ed0 2024-03-16T10:11:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bef7fe640 1 -- 172.21.15.17:0/3831422993 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0014870 con 0x7f5bf8142ed0 2024-03-16T10:11:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/3831422993 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf80b18a0 con 0x7f5bf8142ed0 2024-03-16T10:11:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/3831422993 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf806a130 con 0x7f5bf8142ed0 2024-03-16T10:11:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.525+0000 7f5bef7fe640 1 -- 172.21.15.17:0/3831422993 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f5bf001b020 con 0x7f5bf8142ed0 2024-03-16T10:11:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.529+0000 7f5bef7fe640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5bd8067660 0x7f5bd8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:10.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.529+0000 7f5bef7fe640 1 -- 172.21.15.17:0/3831422993 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5bf0086240 con 0x7f5bf8142ed0 2024-03-16T10:11:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.529+0000 7f5bf659c640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5bd8067660 0x7f5bd8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.529+0000 7f5bf659c640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5bd8067660 0x7f5bd8069b20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5be80ad000 tx=0x7f5be8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.533+0000 7f5bef7fe640 1 -- 172.21.15.17:0/3831422993 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5bf0022070 con 0x7f5bf8142ed0 2024-03-16T10:11:10.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.729+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/3831422993 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5bf8068bb0 con 0x7f5bf8142ed0 2024-03-16T10:11:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.729+0000 7f5bef7fe640 1 -- 172.21.15.17:0/3831422993 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f5bf0010040 con 0x7f5bf8142ed0 2024-03-16T10:11:10.732 INFO:teuthology.orchestra.run.smithi017.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-16T10:11:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.729+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5bd8067660 msgr2=0x7f5bd8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.729+0000 7f5bfcd7a640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5bd8067660 0x7f5bd8069b20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5be80ad000 tx=0x7f5be8008040 comp rx=0 tx=0).stop 2024-03-16T10:11:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.729+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/3831422993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bf8142ed0 msgr2=0x7f5bf81e4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.729+0000 7f5bfcd7a640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bf8142ed0 0x7f5bf81e4260 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0009d30 tx=0x7f5bf000c490 comp rx=0 tx=0).stop 2024-03-16T10:11:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.729+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/3831422993 shutdown_connections 2024-03-16T10:11:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.729+0000 7f5bfcd7a640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5bd8067660 0x7f5bd8069b20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.729+0000 7f5bfcd7a640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5bf8142ed0 0x7f5bf81e4260 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.729+0000 7f5bfcd7a640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5bf80b3cf0 0x7f5bf8144640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.729+0000 7f5bfcd7a640 1 --2- 172.21.15.17:0/3831422993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bf80b5710 0x7f5bf813d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.729+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/3831422993 >> 172.21.15.17:0/3831422993 conn(0x7f5bf80b6800 msgr2=0x7f5bf80afde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.729+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/3831422993 shutdown_connections 2024-03-16T10:11:10.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10.729+0000 7f5bfcd7a640 1 -- 172.21.15.17:0/3831422993 wait complete. 2024-03-16T10:11:10.750 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T10:11:10.750 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T10:11:10.750 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi017.front.sepia.ceph.com 2024-03-16T10:11:10.750 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-16T10:11:10.750 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:10.750 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T10:11:10.750 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T10:11:10.750 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T10:11:10.750 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-16T10:11:10.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:10.750 DEBUG:teuthology.orchestra.run.smithi017:> ip addr 2024-03-16T10:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T10:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T10:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T10:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 ::1/128 scope host 2024-03-16T10:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-16T10:11:10.756 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:00 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f0 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:01 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f1 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:92 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f0 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.21.15.17/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft 42531sec preferred_lft 42531sec 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 fe80::ec4:7aff:fe59:d992/64 scope link 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:93 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f1 2024-03-16T10:11:10.757 INFO:teuthology.orchestra.run.smithi017.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-16T10:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 02:42:d8:82:b9:a0 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-16T10:11:10.758 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:11:10.758 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-16T10:11:10.758 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:11:10.758 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-16T10:11:10.758 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add name ceph-brx type bridge 2024-03-16T10:11:10.758 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr flush dev ceph-brx 2024-03-16T10:11:10.758 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set ceph-brx up 2024-03-16T10:11:10.758 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-16T10:11:10.759 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-16T10:11:10.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:11:10.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:11:10.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:10.891 DEBUG:teuthology.orchestra.run.smithi017:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T10:11:10.899 INFO:teuthology.orchestra.run.smithi017.stdout:1 2024-03-16T10:11:10.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:10.900 DEBUG:teuthology.orchestra.run.smithi017:> route 2024-03-16T10:11:10.963 INFO:teuthology.orchestra.run.smithi017.stdout:Kernel IP routing table 2024-03-16T10:11:10.963 INFO:teuthology.orchestra.run.smithi017.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T10:11:10.963 INFO:teuthology.orchestra.run.smithi017.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-16T10:11:10.963 INFO:teuthology.orchestra.run.smithi017.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-16T10:11:10.963 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-16T10:11:10.963 INFO:teuthology.orchestra.run.smithi017.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T10:11:10.964 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:11:10.964 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-16T10:11:10.964 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-16T10:11:10.964 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-16T10:11:10.964 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-16T10:11:10.964 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-16T10:11:11.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:11:11.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:11:11.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:11.078 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-03-16T10:11:11.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:11.123 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list-id 2024-03-16T10:11:11.170 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:11:11.170 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-16T10:11:11.171 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:11:11.171 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-16T10:11:11.171 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-16T10:11:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:11:11.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:11:11.282 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-16T10:11:11.282 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:11:11.282 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-16T10:11:11.282 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-16T10:11:11.283 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-16T10:11:11.283 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-16T10:11:11.283 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-16T10:11:11.283 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-16T10:11:11.283 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-16T10:11:11.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:11:11.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:11:11.420 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:11:11.420 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-16T10:11:11.420 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set brx.0 up 2024-03-16T10:11:11.420 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set dev brx.0 master ceph-brx 2024-03-16T10:11:11.420 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-16T10:11:11.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:11:11.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:11:11.529 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-16T10:11:11.529 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T10:11:11.529 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:11.583 INFO:teuthology.orchestra.run.smithi017.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-16T10:11:11.584 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T10:11:11.584 DEBUG:teuthology.orchestra.run.smithi017:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:11.635 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-03-16T10:11:11.693 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T10:11:11.740 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-03-16T10:11:11.740 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-03-16T10:11:11.740 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-03-16T10:11:11.740 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/run/lock 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/fs/nfsd 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-03-16T10:11:11.741 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysusers.service 2024-03-16T10:11:11.742 INFO:teuthology.orchestra.run.smithi017.stdout:/run/rpc_pipefs 2024-03-16T10:11:11.742 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-03-16T10:11:11.742 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-03-16T10:11:11.742 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-0 2024-03-16T10:11:11.742 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-1 2024-03-16T10:11:11.742 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-2 2024-03-16T10:11:11.742 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-3 2024-03-16T10:11:11.742 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-03-16T10:11:11.742 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:11:11.742 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:11:11.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:11.743 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-03-16T10:11:11.791 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-16T10:11:11.791 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-16T10:11:11.836 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-03-16T10:11:11.849 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T10:11:11.896 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/run/lock 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-03-16T10:11:11.897 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/fs/nfsd 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysusers.service 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/run/rpc_pipefs 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-0 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-1 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-2 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-3 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:11:11.898 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:11:11.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:11.899 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-03-16T10:11:11.921 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi017.stderr:2024-03-16T10:11:11.917+0000 7f406a862580 -1 init, newargv = 0x5583be019c20 newargc=15 2024-03-16T10:11:11.921 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi017.stderr:ceph-fuse[33615]: starting ceph client 2024-03-16T10:11:11.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi017.stderr:ceph-fuse[33615]: starting fuse 2024-03-16T10:11:13.027 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-03-16T10:11:13.042 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T10:11:13.088 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-03-16T10:11:13.088 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/run/lock 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-03-16T10:11:13.089 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/fs/nfsd 2024-03-16T10:11:13.090 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-03-16T10:11:13.090 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-03-16T10:11:13.090 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysusers.service 2024-03-16T10:11:13.090 INFO:teuthology.orchestra.run.smithi017.stdout:/run/rpc_pipefs 2024-03-16T10:11:13.090 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-03-16T10:11:13.090 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-03-16T10:11:13.090 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-0 2024-03-16T10:11:13.090 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-1 2024-03-16T10:11:13.090 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-2 2024-03-16T10:11:13.090 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-3 2024-03-16T10:11:13.090 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-03-16T10:11:13.090 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:11:13.090 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:11:13.090 INFO:teuthology.orchestra.run.smithi017.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:13.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:13.092 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-03-16T10:11:13.139 INFO:teuthology.orchestra.run.smithi017.stdout:45 2024-03-16T10:11:13.139 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-16T10:11:13.140 DEBUG:teuthology.orchestra.run.smithi017:> sudo stdin-killer -- python3 -c ' 2024-03-16T10:11:13.140 DEBUG:teuthology.orchestra.run.smithi017:> import glob 2024-03-16T10:11:13.140 DEBUG:teuthology.orchestra.run.smithi017:> import re 2024-03-16T10:11:13.140 DEBUG:teuthology.orchestra.run.smithi017:> import os 2024-03-16T10:11:13.140 DEBUG:teuthology.orchestra.run.smithi017:> import subprocess 2024-03-16T10:11:13.140 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-16T10:11:13.140 DEBUG:teuthology.orchestra.run.smithi017:> def _find_admin_socket(client_name): 2024-03-16T10:11:13.140 DEBUG:teuthology.orchestra.run.smithi017:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-16T10:11:13.140 DEBUG:teuthology.orchestra.run.smithi017:> files = glob.glob(asok_path) 2024-03-16T10:11:13.140 DEBUG:teuthology.orchestra.run.smithi017:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-16T10:11:13.140 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-16T10:11:13.140 DEBUG:teuthology.orchestra.run.smithi017:> # Given a non-glob path, it better be there 2024-03-16T10:11:13.140 DEBUG:teuthology.orchestra.run.smithi017:> if "*" not in asok_path: 2024-03-16T10:11:13.140 DEBUG:teuthology.orchestra.run.smithi017:> assert(len(files) == 1) 2024-03-16T10:11:13.141 DEBUG:teuthology.orchestra.run.smithi017:> return files[0] 2024-03-16T10:11:13.141 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-16T10:11:13.141 DEBUG:teuthology.orchestra.run.smithi017:> for f in files: 2024-03-16T10:11:13.141 DEBUG:teuthology.orchestra.run.smithi017:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-16T10:11:13.141 DEBUG:teuthology.orchestra.run.smithi017:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-16T10:11:13.141 DEBUG:teuthology.orchestra.run.smithi017:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-16T10:11:13.141 DEBUG:teuthology.orchestra.run.smithi017:> contents = proc_f.read() 2024-03-16T10:11:13.141 DEBUG:teuthology.orchestra.run.smithi017:> if mountpoint in contents: 2024-03-16T10:11:13.141 DEBUG:teuthology.orchestra.run.smithi017:> return f 2024-03-16T10:11:13.141 DEBUG:teuthology.orchestra.run.smithi017:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-16T10:11:13.141 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-16T10:11:13.141 DEBUG:teuthology.orchestra.run.smithi017:> print(_find_admin_socket("client.0")) 2024-03-16T10:11:13.141 DEBUG:teuthology.orchestra.run.smithi017:> ' 2024-03-16T10:11:13.278 INFO:teuthology.orchestra.run.smithi017.stdout:/var/run/ceph/ceph-client.0.33615.asok 2024-03-16T10:11:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:11:13.290 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33615.asok 2024-03-16T10:11:13.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:13.291 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33615.asok status 2024-03-16T10:11:13.446 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-16T10:11:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": { 2024-03-16T10:11:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_sha1": "3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", 2024-03-16T10:11:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-16T10:11:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "entity_id": "0", 2024-03-16T10:11:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "hostname": "smithi017", 2024-03-16T10:11:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-16T10:11:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "pid": "33615", 2024-03-16T10:11:13.446 INFO:teuthology.orchestra.run.smithi017.stdout: "root": "/" 2024-03-16T10:11:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_count": 0, 2024-03-16T10:11:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_pinned_count": 0, 2024-03-16T10:11:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4632, 2024-03-16T10:11:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "inst": { 2024-03-16T10:11:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "name": { 2024-03-16T10:11:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "client", 2024-03-16T10:11:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4632 2024-03-16T10:11:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2024-03-16T10:11:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-03-16T10:11:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2024-03-16T10:11:13.447 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1406771810 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1406771810 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "inst_str": "client.4632 192.168.0.1:0/1406771810", 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "addr_str": "192.168.0.1:0/1406771810", 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "inode_count": 1, 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_epoch": 8, 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch": 21, 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch_barrier": 0, 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "blocklisted": false, 2024-03-16T10:11:13.448 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs" 2024-03-16T10:11:13.449 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-16T10:11:13.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:13.460 DEBUG:teuthology.orchestra.run.smithi017:> stat --file-system '--printf=%T 2024-03-16T10:11:13.460 DEBUG:teuthology.orchestra.run.smithi017:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:13.466 INFO:teuthology.orchestra.run.smithi017.stdout:fuseblk 2024-03-16T10:11:13.466 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:13.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:13.466 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:13.523 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-16T10:11:13.538 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.685+0000 7fc0af7a1640 1 Processor -- start 2024-03-16T10:11:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.685+0000 7fc0af7a1640 1 -- start start 2024-03-16T10:11:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc0af7a1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc0a806e660 0x7fc0a8150b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc0af7a1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0a806f5a0 con 0x7fc0a80b72e0 2024-03-16T10:11:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc0af7a1640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0a806fdc0 con 0x7fc0a8148a30 2024-03-16T10:11:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc0af7a1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc0a80a0e20 con 0x7fc0a806e660 2024-03-16T10:11:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc0add3e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc0a806e660 0x7fc0a8150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc0acd3c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc0a80b72e0 0x7fc0a8142360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48926/0 (socket says 172.21.15.17:48926) 2024-03-16T10:11:13.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc0acd3c640 1 -- 172.21.15.17:0/2319359400 learned_addr learned my addr 172.21.15.17:0/2319359400 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:13.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc0add3e640 1 -- 172.21.15.17:0/2319359400 >> v1:172.21.15.174:6789/0 conn(0x7fc0a8148a30 legacy=0x7fc0a8148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:13.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc0add3e640 1 -- 172.21.15.17:0/2319359400 >> v1:172.21.15.17:6789/0 conn(0x7fc0a80b72e0 legacy=0x7fc0a8142360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:13.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc09ffff640 1 -- 172.21.15.17:0/2319359400 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1583259353 0 0) 0x7fc0a806f5a0 con 0x7fc0a80b72e0 2024-03-16T10:11:13.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc0add3e640 1 -- 172.21.15.17:0/2319359400 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a8149520 con 0x7fc0a806e660 2024-03-16T10:11:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc0add3e640 1 --2- 172.21.15.17:0/2319359400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a806e660 0x7fc0a8150b10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009d90 tx=0x7fc0a001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47fa3b0ddac5f09 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc09ffff640 1 -- 172.21.15.17:0/2319359400 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00b2070 con 0x7fc0a806e660 2024-03-16T10:11:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc09ffff640 1 -- 172.21.15.17:0/2319359400 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0a00a5710 con 0x7fc0a806e660 2024-03-16T10:11:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.689+0000 7fc09ffff640 1 -- 172.21.15.17:0/2319359400 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00a5ae0 con 0x7fc0a806e660 2024-03-16T10:11:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 -- 172.21.15.17:0/2319359400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a806e660 msgr2=0x7fc0a8150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 --2- 172.21.15.17:0/2319359400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a806e660 0x7fc0a8150b10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0009d90 tx=0x7fc0a001c810 comp rx=0 tx=0).stop 2024-03-16T10:11:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 -- 172.21.15.17:0/2319359400 shutdown_connections 2024-03-16T10:11:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 --2- 172.21.15.17:0/2319359400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a806e660 0x7fc0a8150b10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 -- 172.21.15.17:0/2319359400 >> 172.21.15.17:0/2319359400 conn(0x7fc0a80718a0 msgr2=0x7fc0a8071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 -- 172.21.15.17:0/2319359400 shutdown_connections 2024-03-16T10:11:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 -- 172.21.15.17:0/2319359400 wait complete. 2024-03-16T10:11:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 Processor -- start 2024-03-16T10:11:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 -- start start 2024-03-16T10:11:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0a806e660 0x7fc0a80b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a8148a30 0x7fc0a80b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0a8143640 0x7fc0a80b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a81496f0 con 0x7fc0a8148a30 2024-03-16T10:11:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0a80a0fa0 con 0x7fc0a8143640 2024-03-16T10:11:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc0a8153a80 con 0x7fc0a806e660 2024-03-16T10:11:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0acd3c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a8148a30 0x7fc0a80b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0acd3c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a8148a30 0x7fc0a80b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55606/0 (socket says 172.21.15.17:55606) 2024-03-16T10:11:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0acd3c640 1 -- 172.21.15.17:0/931667952 learned_addr learned my addr 172.21.15.17:0/931667952 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0ad53d640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0a806e660 0x7fc0a80b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0add3e640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0a8143640 0x7fc0a80b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0acd3c640 1 -- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0a806e660 msgr2=0x7fc0a80b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0acd3c640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0a806e660 0x7fc0a80b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0acd3c640 1 -- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0a8143640 msgr2=0x7fc0a80b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0acd3c640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0a8143640 0x7fc0a80b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:13.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0acd3c640 1 -- 172.21.15.17:0/931667952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0a80bab90 con 0x7fc0a8148a30 2024-03-16T10:11:13.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0ad53d640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0a806e660 0x7fc0a80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:13.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0add3e640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0a8143640 0x7fc0a80b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:13.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0acd3c640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a8148a30 0x7fc0a80b43c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc09800c480 tx=0x7fc09800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:13.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc09dffb640 1 -- 172.21.15.17:0/931667952 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098018070 con 0x7fc0a8148a30 2024-03-16T10:11:13.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc09dffb640 1 -- 172.21.15.17:0/931667952 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0980074e0 con 0x7fc0a8148a30 2024-03-16T10:11:13.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 -- 172.21.15.17:0/931667952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0a8143ad0 con 0x7fc0a8148a30 2024-03-16T10:11:13.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc09dffb640 1 -- 172.21.15.17:0/931667952 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc098004a30 con 0x7fc0a8148a30 2024-03-16T10:11:13.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.693+0000 7fc0af7a1640 1 -- 172.21.15.17:0/931667952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0a80b4aa0 con 0x7fc0a8148a30 2024-03-16T10:11:13.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.697+0000 7fc0af7a1640 1 -- 172.21.15.17:0/931667952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0a806a130 con 0x7fc0a8148a30 2024-03-16T10:11:13.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.697+0000 7fc09dffb640 1 -- 172.21.15.17:0/931667952 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fc09801b070 con 0x7fc0a8148a30 2024-03-16T10:11:13.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.697+0000 7fc09dffb640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc08c067660 0x7fc08c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:13.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.697+0000 7fc09dffb640 1 -- 172.21.15.17:0/931667952 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc098084b20 con 0x7fc0a8148a30 2024-03-16T10:11:13.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.697+0000 7fc0ad53d640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc08c067660 0x7fc08c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:13.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.697+0000 7fc0ad53d640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc08c067660 0x7fc08c069b20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80b51a0 tx=0x7fc0a4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:13.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.701+0000 7fc09dffb640 1 -- 172.21.15.17:0/931667952 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc098055830 con 0x7fc0a8148a30 2024-03-16T10:11:13.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.861+0000 7fc0af7a1640 1 -- 172.21.15.17:0/931667952 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc0a813af80 con 0x7fc0a8148a30 2024-03-16T10:11:13.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.865+0000 7fc09dffb640 1 -- 172.21.15.17:0/931667952 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7fc098055130 con 0x7fc0a8148a30 2024-03-16T10:11:13.868 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:13.869 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.865+0000 7fc0af7a1640 1 -- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc08c067660 msgr2=0x7fc08c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.865+0000 7fc0af7a1640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc08c067660 0x7fc08c069b20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc0a80b51a0 tx=0x7fc0a4009040 comp rx=0 tx=0).stop 2024-03-16T10:11:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.865+0000 7fc0af7a1640 1 -- 172.21.15.17:0/931667952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a8148a30 msgr2=0x7fc0a80b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.865+0000 7fc0af7a1640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a8148a30 0x7fc0a80b43c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc09800c480 tx=0x7fc09800c940 comp rx=0 tx=0).stop 2024-03-16T10:11:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.865+0000 7fc0af7a1640 1 -- 172.21.15.17:0/931667952 shutdown_connections 2024-03-16T10:11:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.865+0000 7fc0af7a1640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc08c067660 0x7fc08c069b20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.865+0000 7fc0af7a1640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0a8143640 0x7fc0a80b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:13.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.865+0000 7fc0af7a1640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0a8148a30 0x7fc0a80b43c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:13.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.865+0000 7fc0af7a1640 1 --2- 172.21.15.17:0/931667952 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc0a806e660 0x7fc0a80b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:13.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.865+0000 7fc0af7a1640 1 -- 172.21.15.17:0/931667952 >> 172.21.15.17:0/931667952 conn(0x7fc0a80718a0 msgr2=0x7fc0a8136b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:13.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.865+0000 7fc0af7a1640 1 -- 172.21.15.17:0/931667952 shutdown_connections 2024-03-16T10:11:13.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:13.865+0000 7fc0af7a1640 1 -- 172.21.15.17:0/931667952 wait complete. 2024-03-16T10:11:13.895 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:10:55.723734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:10:56.459894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:11:13.895 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:11:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca7cbe640 1 Processor -- start 2024-03-16T10:11:14.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca7cbe640 1 -- start start 2024-03-16T10:11:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca7cbe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ca0067220 0x7f3ca00b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca7cbe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ca00b3da0 0x7f3ca0147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca7cbe640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ca00b05a0 con 0x7f3ca00b4200 2024-03-16T10:11:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca7cbe640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3ca006e4d0 con 0x7f3ca00b3da0 2024-03-16T10:11:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca7cbe640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca0067600 con 0x7f3ca0067220 2024-03-16T10:11:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca5259640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ca00b3da0 0x7f3ca0147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca5259640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ca00b3da0 0x7f3ca0147fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55616/0 (socket says 172.21.15.17:55616) 2024-03-16T10:11:14.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca5259640 1 -- 172.21.15.17:0/2691936289 learned_addr learned my addr 172.21.15.17:0/2691936289 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:14.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca5a5a640 1 --2- 172.21.15.17:0/2691936289 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ca0067220 0x7f3ca00b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca5259640 1 -- 172.21.15.17:0/2691936289 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ca0067220 msgr2=0x7f3ca00b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:14.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca5259640 1 --2- 172.21.15.17:0/2691936289 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ca0067220 0x7f3ca00b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca5259640 1 -- 172.21.15.17:0/2691936289 >> v1:172.21.15.174:6789/0 conn(0x7f3ca00b4200 legacy=0x7f3ca014ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca5259640 1 -- 172.21.15.17:0/2691936289 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca006e1f0 con 0x7f3ca00b3da0 2024-03-16T10:11:14.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.021+0000 7f3ca5a5a640 1 --2- 172.21.15.17:0/2691936289 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ca0067220 0x7f3ca00b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:14.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca5259640 1 --2- 172.21.15.17:0/2691936289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca00b3da0 0x7f3ca0147fc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009870 tx=0x7f3c9c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322281b026574399 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:14.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca4a58640 1 -- 172.21.15.17:0/2691936289 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0b2070 con 0x7f3ca00b3da0 2024-03-16T10:11:14.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca4a58640 1 -- 172.21.15.17:0/2691936289 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c9c0a9070 con 0x7f3ca00b3da0 2024-03-16T10:11:14.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca4a58640 1 -- 172.21.15.17:0/2691936289 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0ad880 con 0x7f3ca00b3da0 2024-03-16T10:11:14.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/2691936289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca00b3da0 msgr2=0x7f3ca0147fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 --2- 172.21.15.17:0/2691936289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca00b3da0 0x7f3ca0147fc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009870 tx=0x7f3c9c01c560 comp rx=0 tx=0).stop 2024-03-16T10:11:14.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/2691936289 shutdown_connections 2024-03-16T10:11:14.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 --2- 172.21.15.17:0/2691936289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca00b3da0 0x7f3ca0147fc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 --2- 172.21.15.17:0/2691936289 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ca0067220 0x7f3ca00b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/2691936289 >> 172.21.15.17:0/2691936289 conn(0x7f3ca00b5ea0 msgr2=0x7f3ca00b6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:14.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/2691936289 shutdown_connections 2024-03-16T10:11:14.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/2691936289 wait complete. 2024-03-16T10:11:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 Processor -- start 2024-03-16T10:11:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 -- start start 2024-03-16T10:11:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ca0067220 0x7f3ca0147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca00b3da0 0x7f3ca01e0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ca00b4200 0x7f3ca009c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca00aeba0 con 0x7f3ca00b3da0 2024-03-16T10:11:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca00aea20 con 0x7f3ca00b4200 2024-03-16T10:11:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca0155060 con 0x7f3ca0067220 2024-03-16T10:11:14.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca5259640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca00b3da0 0x7f3ca01e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca5259640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca00b3da0 0x7f3ca01e0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55630/0 (socket says 172.21.15.17:55630) 2024-03-16T10:11:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca5259640 1 -- 172.21.15.17:0/1528001444 learned_addr learned my addr 172.21.15.17:0/1528001444 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca5a5a640 1 --2- 172.21.15.17:0/1528001444 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ca0067220 0x7f3ca0147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca625b640 1 --2- 172.21.15.17:0/1528001444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ca00b4200 0x7f3ca009c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca5259640 1 -- 172.21.15.17:0/1528001444 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ca0067220 msgr2=0x7f3ca0147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca5259640 1 --2- 172.21.15.17:0/1528001444 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ca0067220 0x7f3ca0147260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca5259640 1 -- 172.21.15.17:0/1528001444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ca00b4200 msgr2=0x7f3ca009c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca5259640 1 --2- 172.21.15.17:0/1528001444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ca00b4200 0x7f3ca009c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca5259640 1 -- 172.21.15.17:0/1528001444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca0071ff0 con 0x7f3ca00b3da0 2024-03-16T10:11:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca5259640 1 --2- 172.21.15.17:0/1528001444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca00b3da0 0x7f3ca01e0490 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0b0040 tx=0x7f3c9c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3c927fc640 1 -- 172.21.15.17:0/1528001444 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0bc020 con 0x7f3ca00b3da0 2024-03-16T10:11:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3c927fc640 1 -- 172.21.15.17:0/1528001444 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c9c0b8420 con 0x7f3ca00b3da0 2024-03-16T10:11:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/1528001444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca009d380 con 0x7f3ca00b3da0 2024-03-16T10:11:14.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.025+0000 7f3c927fc640 1 -- 172.21.15.17:0/1528001444 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9c0b2040 con 0x7f3ca00b3da0 2024-03-16T10:11:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.029+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/1528001444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca009e3a0 con 0x7f3ca00b3da0 2024-03-16T10:11:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.029+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/1528001444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca006a130 con 0x7f3ca00b3da0 2024-03-16T10:11:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.029+0000 7f3c927fc640 1 -- 172.21.15.17:0/1528001444 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f3c9c0bf020 con 0x7f3ca00b3da0 2024-03-16T10:11:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.029+0000 7f3c927fc640 1 --2- 172.21.15.17:0/1528001444 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3c84067660 0x7f3c84069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.029+0000 7f3c927fc640 1 -- 172.21.15.17:0/1528001444 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3c9c11fb10 con 0x7f3ca00b3da0 2024-03-16T10:11:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.029+0000 7f3ca5a5a640 1 --2- 172.21.15.17:0/1528001444 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3c84067660 0x7f3c84069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.029+0000 7f3ca5a5a640 1 --2- 172.21.15.17:0/1528001444 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3c84067660 0x7f3c84069b20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3c98008ee0 tx=0x7f3c98008430 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:14.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.033+0000 7f3c927fc640 1 -- 172.21.15.17:0/1528001444 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3c9c0c6030 con 0x7f3ca00b3da0 2024-03-16T10:11:14.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.232+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/1528001444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3ca00722d0 con 0x7f3ca00b3da0 2024-03-16T10:11:14.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.232+0000 7f3c927fc640 1 -- 172.21.15.17:0/1528001444 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f3c9c016150 con 0x7f3ca00b3da0 2024-03-16T10:11:14.239 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:11:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.236+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/1528001444 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3c84067660 msgr2=0x7f3c84069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.236+0000 7f3ca7cbe640 1 --2- 172.21.15.17:0/1528001444 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3c84067660 0x7f3c84069b20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3c98008ee0 tx=0x7f3c98008430 comp rx=0 tx=0).stop 2024-03-16T10:11:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.236+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/1528001444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca00b3da0 msgr2=0x7f3ca01e0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.236+0000 7f3ca7cbe640 1 --2- 172.21.15.17:0/1528001444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca00b3da0 0x7f3ca01e0490 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0b0040 tx=0x7f3c9c01ada0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.236+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/1528001444 shutdown_connections 2024-03-16T10:11:14.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.236+0000 7f3ca7cbe640 1 --2- 172.21.15.17:0/1528001444 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3c84067660 0x7f3c84069b20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.236+0000 7f3ca7cbe640 1 --2- 172.21.15.17:0/1528001444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ca00b4200 0x7f3ca009c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.236+0000 7f3ca7cbe640 1 --2- 172.21.15.17:0/1528001444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ca00b3da0 0x7f3ca01e0490 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.236+0000 7f3ca7cbe640 1 --2- 172.21.15.17:0/1528001444 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ca0067220 0x7f3ca0147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.236+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/1528001444 >> 172.21.15.17:0/1528001444 conn(0x7f3ca00b5ea0 msgr2=0x7f3ca00b6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:14.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.236+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/1528001444 shutdown_connections 2024-03-16T10:11:14.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.236+0000 7f3ca7cbe640 1 -- 172.21.15.17:0/1528001444 wait complete. 2024-03-16T10:11:14.260 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:11:14.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43c5501640 1 Processor -- start 2024-03-16T10:11:14.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43c5501640 1 -- start start 2024-03-16T10:11:14.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43c5501640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f43c014b230 0x7f43c0144480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43c5501640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f43c01449c0 0x7f43c0144e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43c5501640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c0067600 con 0x7f43c0067220 2024-03-16T10:11:14.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43c5501640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f43c00b46f0 con 0x7f43c014b230 2024-03-16T10:11:14.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43c5501640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f43c00b57f0 con 0x7f43c01449c0 2024-03-16T10:11:14.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43be7fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f43c014b230 0x7f43c0144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43beffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f43c0067220 0x7f43c014ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48940/0 (socket says 172.21.15.17:48940) 2024-03-16T10:11:14.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43beffd640 1 -- 172.21.15.17:0/1478366536 learned_addr learned my addr 172.21.15.17:0/1478366536 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:14.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43bdffb640 1 -- 172.21.15.17:0/1478366536 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3376728334 0 0) 0x7f43c0067600 con 0x7f43c0067220 2024-03-16T10:11:14.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43bdffb640 1 -- 172.21.15.17:0/1478366536 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43a0003530 con 0x7f43c0067220 2024-03-16T10:11:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43bf7fe640 1 --2- 172.21.15.17:0/1478366536 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f43c01449c0 0x7f43c0144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43be7fc640 1 -- 172.21.15.17:0/1478366536 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f43c01449c0 msgr2=0x7f43c0144e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43be7fc640 1 --2- 172.21.15.17:0/1478366536 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f43c01449c0 0x7f43c0144e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43be7fc640 1 -- 172.21.15.17:0/1478366536 >> v1:172.21.15.17:6789/0 conn(0x7f43c0067220 legacy=0x7f43c014ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43be7fc640 1 -- 172.21.15.17:0/1478366536 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c00b4080 con 0x7f43c014b230 2024-03-16T10:11:14.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.380+0000 7f43bf7fe640 1 --2- 172.21.15.17:0/1478366536 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f43c01449c0 0x7f43c0144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:14.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.384+0000 7f43be7fc640 1 --2- 172.21.15.17:0/1478366536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c014b230 0x7f43c0144480 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009d90 tx=0x7f43b001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6fedca2b0d2f0f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:14.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.384+0000 7f43bdffb640 1 -- 172.21.15.17:0/1478366536 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b00b2070 con 0x7f43c014b230 2024-03-16T10:11:14.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.384+0000 7f43bdffb640 1 -- 172.21.15.17:0/1478366536 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b00a58b0 con 0x7f43c014b230 2024-03-16T10:11:14.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.384+0000 7f43bdffb640 1 -- 172.21.15.17:0/1478366536 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b00a5c80 con 0x7f43c014b230 2024-03-16T10:11:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.384+0000 7f43c5501640 1 -- 172.21.15.17:0/1478366536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c014b230 msgr2=0x7f43c0144480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.384+0000 7f43c5501640 1 --2- 172.21.15.17:0/1478366536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c014b230 0x7f43c0144480 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009d90 tx=0x7f43b001c8e0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.384+0000 7f43c5501640 1 -- 172.21.15.17:0/1478366536 shutdown_connections 2024-03-16T10:11:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.384+0000 7f43c5501640 1 --2- 172.21.15.17:0/1478366536 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f43c01449c0 0x7f43c0144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.384+0000 7f43c5501640 1 --2- 172.21.15.17:0/1478366536 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c014b230 0x7f43c0144480 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.384+0000 7f43c5501640 1 -- 172.21.15.17:0/1478366536 >> 172.21.15.17:0/1478366536 conn(0x7f43c006bd70 msgr2=0x7f43c00b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:14.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.384+0000 7f43c5501640 1 -- 172.21.15.17:0/1478366536 shutdown_connections 2024-03-16T10:11:14.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.384+0000 7f43c5501640 1 -- 172.21.15.17:0/1478366536 wait complete. 2024-03-16T10:11:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.384+0000 7f43c5501640 1 Processor -- start 2024-03-16T10:11:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43c5501640 1 -- start start 2024-03-16T10:11:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43c5501640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f43c0067220 0x7f43c0143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43c5501640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43c01449c0 0x7f43c01dff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43c5501640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c014b230 0x7f43c0146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43c5501640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f43c00b3e20 con 0x7f43c014b230 2024-03-16T10:11:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43c5501640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f43c00b3ca0 con 0x7f43c01449c0 2024-03-16T10:11:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43c5501640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f43c0154880 con 0x7f43c0067220 2024-03-16T10:11:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43bf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c014b230 0x7f43c0146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43bf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c014b230 0x7f43c0146830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55640/0 (socket says 172.21.15.17:55640) 2024-03-16T10:11:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43bf7fe640 1 -- 172.21.15.17:0/2626379494 learned_addr learned my addr 172.21.15.17:0/2626379494 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:14.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43be7fc640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43c01449c0 0x7f43c01dff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43bf7fe640 1 -- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f43c0067220 msgr2=0x7f43c0143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43beffd640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f43c0067220 0x7f43c0143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43bf7fe640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f43c0067220 0x7f43c0143c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43bf7fe640 1 -- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43c01449c0 msgr2=0x7f43c01dff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43bf7fe640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43c01449c0 0x7f43c01dff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43bf7fe640 1 -- 172.21.15.17:0/2626379494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c0072d60 con 0x7f43c014b230 2024-03-16T10:11:14.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43be7fc640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43c01449c0 0x7f43c01dff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:14.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43bf7fe640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c014b230 0x7f43c0146830 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f43b800c7f0 tx=0x7f43b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:14.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43b77fe640 1 -- 172.21.15.17:0/2626379494 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8015070 con 0x7f43c014b230 2024-03-16T10:11:14.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43b77fe640 1 -- 172.21.15.17:0/2626379494 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b8007860 con 0x7f43c014b230 2024-03-16T10:11:14.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43b77fe640 1 -- 172.21.15.17:0/2626379494 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8007c50 con 0x7f43c014b230 2024-03-16T10:11:14.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43c5501640 1 -- 172.21.15.17:0/2626379494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c0147af0 con 0x7f43c014b230 2024-03-16T10:11:14.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43c5501640 1 -- 172.21.15.17:0/2626379494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c0146310 con 0x7f43c014b230 2024-03-16T10:11:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43c5501640 1 -- 172.21.15.17:0/2626379494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c006a130 con 0x7f43c014b230 2024-03-16T10:11:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43b77fe640 1 -- 172.21.15.17:0/2626379494 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f43b8018070 con 0x7f43c014b230 2024-03-16T10:11:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.388+0000 7f43b77fe640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f43a4067660 0x7f43a4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.392+0000 7f43b77fe640 1 -- 172.21.15.17:0/2626379494 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f43b8085a00 con 0x7f43c014b230 2024-03-16T10:11:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.392+0000 7f43beffd640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f43a4067660 0x7f43a4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.392+0000 7f43beffd640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f43a4067660 0x7f43a4069b20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f43a8007fd0 tx=0x7f43a8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:14.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.396+0000 7f43b77fe640 1 -- 172.21.15.17:0/2626379494 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f43b8056710 con 0x7f43c014b230 2024-03-16T10:11:14.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.732+0000 7f43c5501640 1 -- 172.21.15.17:0/2626379494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f43c009af50 con 0x7f43c014b230 2024-03-16T10:11:14.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.732+0000 7f43b77fe640 1 -- 172.21.15.17:0/2626379494 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f43b8056010 con 0x7f43c014b230 2024-03-16T10:11:14.738 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-16T10:11:14.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.736+0000 7f43c5501640 1 -- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f43a4067660 msgr2=0x7f43a4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.736+0000 7f43c5501640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f43a4067660 0x7f43a4069b20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f43a8007fd0 tx=0x7f43a8009040 comp rx=0 tx=0).stop 2024-03-16T10:11:14.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.736+0000 7f43c5501640 1 -- 172.21.15.17:0/2626379494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c014b230 msgr2=0x7f43c0146830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.736+0000 7f43c5501640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c014b230 0x7f43c0146830 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f43b800c7f0 tx=0x7f43b800ccb0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.736+0000 7f43c5501640 1 -- 172.21.15.17:0/2626379494 shutdown_connections 2024-03-16T10:11:14.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.736+0000 7f43c5501640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f43a4067660 0x7f43a4069b20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.736+0000 7f43c5501640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43c014b230 0x7f43c0146830 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.736+0000 7f43c5501640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43c01449c0 0x7f43c01dff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.736+0000 7f43c5501640 1 --2- 172.21.15.17:0/2626379494 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f43c0067220 0x7f43c0143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.736+0000 7f43c5501640 1 -- 172.21.15.17:0/2626379494 >> 172.21.15.17:0/2626379494 conn(0x7f43c006bd70 msgr2=0x7f43c00ae560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:14.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.736+0000 7f43c5501640 1 -- 172.21.15.17:0/2626379494 shutdown_connections 2024-03-16T10:11:14.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.736+0000 7f43c5501640 1 -- 172.21.15.17:0/2626379494 wait complete. 2024-03-16T10:11:14.758 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:11:14.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.872+0000 7f67cb76d640 1 Processor -- start 2024-03-16T10:11:14.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.872+0000 7f67cb76d640 1 -- start start 2024-03-16T10:11:14.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67cb76d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67c40aeaf0 0x7f67c4071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67cb76d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f67c4072410 0x7f67c414e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67cb76d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67c40b5b20 con 0x7f67c40b5740 2024-03-16T10:11:14.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67cb76d640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f67c406e950 con 0x7f67c4072410 2024-03-16T10:11:14.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67cb76d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f67c40aef30 con 0x7f67c40aeaf0 2024-03-16T10:11:14.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c9509640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f67c40b5740 0x7f67c406ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48950/0 (socket says 172.21.15.17:48950) 2024-03-16T10:11:14.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c9509640 1 -- 172.21.15.17:0/321815049 learned_addr learned my addr 172.21.15.17:0/321815049 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:14.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 -- 172.21.15.17:0/321815049 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 993774786 0 0) 0x7f67c40b5b20 con 0x7f67c40b5740 2024-03-16T10:11:14.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c9d0a640 1 --2- 172.21.15.17:0/321815049 >> v2:172.21.15.174:3300/0 conn(0x7f67c4072410 0x7f67c414e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c8d08640 1 --2- 172.21.15.17:0/321815049 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67c40aeaf0 0x7f67c4071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 -- 172.21.15.17:0/321815049 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f67ac003530 con 0x7f67c40b5740 2024-03-16T10:11:14.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 -- 172.21.15.17:0/321815049 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3901604497 0 0) 0x7f67ac003530 con 0x7f67c40b5740 2024-03-16T10:11:14.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 -- 172.21.15.17:0/321815049 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67c40aeaf0 msgr2=0x7f67c4071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:14.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 --2- 172.21.15.17:0/321815049 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67c40aeaf0 0x7f67c4071ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 -- 172.21.15.17:0/321815049 >> v2:172.21.15.174:3300/0 conn(0x7f67c4072410 msgr2=0x7f67c414e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:14.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 --2- 172.21.15.17:0/321815049 >> v2:172.21.15.174:3300/0 conn(0x7f67c4072410 0x7f67c414e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 -- 172.21.15.17:0/321815049 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c40af720 con 0x7f67c40b5740 2024-03-16T10:11:14.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 -- 172.21.15.17:0/321815049 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f67bc001680 con 0x7f67c40b5740 2024-03-16T10:11:14.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 -- 172.21.15.17:0/321815049 >> v1:172.21.15.17:6789/0 conn(0x7f67c40b5740 legacy=0x7f67c406ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 --2- 172.21.15.17:0/321815049 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67c40aeaf0 0x7f67ac0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 --2- 172.21.15.17:0/321815049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67ac003f90 0x7f67ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 --2- 172.21.15.17:0/321815049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67ac004d50 0x7f67ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 -- 172.21.15.17:0/321815049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f67c406e950 con 0x7f67ac004d50 2024-03-16T10:11:14.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 -- 172.21.15.17:0/321815049 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f67c40aef30 con 0x7f67ac003f90 2024-03-16T10:11:14.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c3fff640 1 -- 172.21.15.17:0/321815049 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f67ac00b9d0 con 0x7f67c40aeaf0 2024-03-16T10:11:14.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c9d0a640 1 --2- 172.21.15.17:0/321815049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67ac004d50 0x7f67ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c8d08640 1 --2- 172.21.15.17:0/321815049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67ac003f90 0x7f67ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c9509640 1 --2- 172.21.15.17:0/321815049 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67c40aeaf0 0x7f67ac0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c9d0a640 1 -- 172.21.15.17:0/321815049 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67c40aeaf0 msgr2=0x7f67ac0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:14.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c9d0a640 1 --2- 172.21.15.17:0/321815049 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67c40aeaf0 0x7f67ac0039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c9d0a640 1 -- 172.21.15.17:0/321815049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67ac003f90 msgr2=0x7f67ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:14.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c9d0a640 1 --2- 172.21.15.17:0/321815049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67ac003f90 0x7f67ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c9d0a640 1 -- 172.21.15.17:0/321815049 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67ac003530 con 0x7f67ac004d50 2024-03-16T10:11:14.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.876+0000 7f67c8d08640 1 --2- 172.21.15.17:0/321815049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67ac003f90 0x7f67ac004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:14.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67c9d0a640 1 --2- 172.21.15.17:0/321815049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67ac004d50 0x7f67ac0051d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f67b400e170 tx=0x7f67b401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0f8ef1d0a7863e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:14.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67c3fff640 1 -- 172.21.15.17:0/321815049 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b40b2070 con 0x7f67ac004d50 2024-03-16T10:11:14.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67c3fff640 1 -- 172.21.15.17:0/321815049 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b40a5cb0 con 0x7f67ac004d50 2024-03-16T10:11:14.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67c3fff640 1 -- 172.21.15.17:0/321815049 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b4009c10 con 0x7f67ac004d50 2024-03-16T10:11:14.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67cb76d640 1 -- 172.21.15.17:0/321815049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67ac004d50 msgr2=0x7f67ac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67cb76d640 1 --2- 172.21.15.17:0/321815049 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67ac004d50 0x7f67ac0051d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f67b400e170 tx=0x7f67b401cca0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67c9d0a640 1 -- 172.21.15.17:0/321815049 reap_dead start 2024-03-16T10:11:14.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67cb76d640 1 -- 172.21.15.17:0/321815049 shutdown_connections 2024-03-16T10:11:14.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67cb76d640 1 -- 172.21.15.17:0/321815049 >> 172.21.15.17:0/321815049 conn(0x7f67c40b65d0 msgr2=0x7f67c40b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:14.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67cb76d640 1 -- 172.21.15.17:0/321815049 shutdown_connections 2024-03-16T10:11:14.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67cb76d640 1 -- 172.21.15.17:0/321815049 wait complete. 2024-03-16T10:11:14.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67cb76d640 1 Processor -- start 2024-03-16T10:11:14.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67cb76d640 1 -- start start 2024-03-16T10:11:14.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67cb76d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67c41d86d0 0x7f67c4142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67cb76d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67c41d8ab0 0x7f67c4142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67cb76d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67c40b2f00 0x7f67c40b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67cb76d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4072d80 con 0x7f67c40b2f00 2024-03-16T10:11:14.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67cb76d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f67c4072c00 con 0x7f67c41d8ab0 2024-03-16T10:11:14.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67cb76d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f67c40af360 con 0x7f67c41d86d0 2024-03-16T10:11:14.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67c9d0a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67c40b2f00 0x7f67c40b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67c9d0a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67c40b2f00 0x7f67c40b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55658/0 (socket says 172.21.15.17:55658) 2024-03-16T10:11:14.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.880+0000 7f67c9d0a640 1 -- 172.21.15.17:0/2594978221 learned_addr learned my addr 172.21.15.17:0/2594978221 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c8d08640 1 --2- 172.21.15.17:0/2594978221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67c41d8ab0 0x7f67c4142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c9d0a640 1 -- 172.21.15.17:0/2594978221 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67c41d86d0 msgr2=0x7f67c4142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c9d0a640 1 --2- 172.21.15.17:0/2594978221 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67c41d86d0 0x7f67c4142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c9d0a640 1 -- 172.21.15.17:0/2594978221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67c41d8ab0 msgr2=0x7f67c4142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c9d0a640 1 --2- 172.21.15.17:0/2594978221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67c41d8ab0 0x7f67c4142d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c9d0a640 1 -- 172.21.15.17:0/2594978221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c41d8fb0 con 0x7f67c40b2f00 2024-03-16T10:11:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c9d0a640 1 --2- 172.21.15.17:0/2594978221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67c40b2f00 0x7f67c40b3390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f67b400e170 tx=0x7f67b4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c1ffb640 1 -- 172.21.15.17:0/2594978221 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b40a5e60 con 0x7f67c40b2f00 2024-03-16T10:11:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c1ffb640 1 -- 172.21.15.17:0/2594978221 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67b40adda0 con 0x7f67c40b2f00 2024-03-16T10:11:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67cb76d640 1 -- 172.21.15.17:0/2594978221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c41d9290 con 0x7f67c40b2f00 2024-03-16T10:11:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c1ffb640 1 -- 172.21.15.17:0/2594978221 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b40b2040 con 0x7f67c40b2f00 2024-03-16T10:11:14.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67cb76d640 1 -- 172.21.15.17:0/2594978221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c4144e10 con 0x7f67c40b2f00 2024-03-16T10:11:14.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67cb76d640 1 -- 172.21.15.17:0/2594978221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c406a130 con 0x7f67c40b2f00 2024-03-16T10:11:14.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c1ffb640 1 -- 172.21.15.17:0/2594978221 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f67b40b4420 con 0x7f67c40b2f00 2024-03-16T10:11:14.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c1ffb640 1 --2- 172.21.15.17:0/2594978221 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f67b0067660 0x7f67b0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:14.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c1ffb640 1 -- 172.21.15.17:0/2594978221 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f67b4121e80 con 0x7f67c40b2f00 2024-03-16T10:11:14.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c9509640 1 --2- 172.21.15.17:0/2594978221 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f67b0067660 0x7f67b0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:14.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.884+0000 7f67c9509640 1 --2- 172.21.15.17:0/2594978221 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f67b0067660 0x7f67b0069b20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0017f0 tx=0x7f67bc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:14.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:14.892+0000 7f67c1ffb640 1 -- 172.21.15.17:0/2594978221 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f67b4016a30 con 0x7f67c40b2f00 2024-03-16T10:11:15.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.052+0000 7f67cb76d640 1 -- 172.21.15.17:0/2594978221 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f67c4070b00 con 0x7f67c40b2f00 2024-03-16T10:11:15.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.052+0000 7f67c1ffb640 1 -- 172.21.15.17:0/2594978221 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f67b40f2520 con 0x7f67c40b2f00 2024-03-16T10:11:15.058 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-16T10:11:15.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.056+0000 7f67cb76d640 1 -- 172.21.15.17:0/2594978221 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f67b0067660 msgr2=0x7f67b0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.056+0000 7f67cb76d640 1 --2- 172.21.15.17:0/2594978221 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f67b0067660 0x7f67b0069b20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0017f0 tx=0x7f67bc009040 comp rx=0 tx=0).stop 2024-03-16T10:11:15.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.056+0000 7f67cb76d640 1 -- 172.21.15.17:0/2594978221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67c40b2f00 msgr2=0x7f67c40b3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.056+0000 7f67cb76d640 1 --2- 172.21.15.17:0/2594978221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67c40b2f00 0x7f67c40b3390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f67b400e170 tx=0x7f67b4009750 comp rx=0 tx=0).stop 2024-03-16T10:11:15.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.056+0000 7f67cb76d640 1 -- 172.21.15.17:0/2594978221 shutdown_connections 2024-03-16T10:11:15.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.056+0000 7f67cb76d640 1 --2- 172.21.15.17:0/2594978221 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f67b0067660 0x7f67b0069b20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.056+0000 7f67cb76d640 1 --2- 172.21.15.17:0/2594978221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67c40b2f00 0x7f67c40b3390 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.056+0000 7f67cb76d640 1 --2- 172.21.15.17:0/2594978221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67c41d8ab0 0x7f67c4142d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.056+0000 7f67cb76d640 1 --2- 172.21.15.17:0/2594978221 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f67c41d86d0 0x7f67c4142820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.056+0000 7f67cb76d640 1 -- 172.21.15.17:0/2594978221 >> 172.21.15.17:0/2594978221 conn(0x7f67c40b65d0 msgr2=0x7f67c406ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:15.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.056+0000 7f67cb76d640 1 -- 172.21.15.17:0/2594978221 shutdown_connections 2024-03-16T10:11:15.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.056+0000 7f67cb76d640 1 -- 172.21.15.17:0/2594978221 wait complete. 2024-03-16T10:11:15.076 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.180+0000 7f8eedb60640 1 Processor -- start 2024-03-16T10:11:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8eedb60640 1 -- start start 2024-03-16T10:11:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8eedb60640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ee80b48f0 0x7f8ee81487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8eedb60640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ee8073040 con 0x7f8ee80b4da0 2024-03-16T10:11:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8eedb60640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ee813f990 con 0x7f8ee8072c60 2024-03-16T10:11:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8eedb60640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8ee8148e50 con 0x7f8ee80b48f0 2024-03-16T10:11:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8ee7fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8ee80b4da0 0x7f8ee814fb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48960/0 (socket says 172.21.15.17:48960) 2024-03-16T10:11:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8ee7fff640 1 -- 172.21.15.17:0/1669210160 learned_addr learned my addr 172.21.15.17:0/1669210160 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8ee6ffd640 1 --2- 172.21.15.17:0/1669210160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ee80b48f0 0x7f8ee81487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8ee67fc640 1 -- 172.21.15.17:0/1669210160 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081400524 0 0) 0x7f8ee8073040 con 0x7f8ee80b4da0 2024-03-16T10:11:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8ee67fc640 1 -- 172.21.15.17:0/1669210160 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ed0003530 con 0x7f8ee80b4da0 2024-03-16T10:11:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8ee67fc640 1 -- 172.21.15.17:0/1669210160 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 294339886 0 0) 0x7f8ee813f990 con 0x7f8ee8072c60 2024-03-16T10:11:15.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8ee67fc640 1 -- 172.21.15.17:0/1669210160 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ee8073040 con 0x7f8ee8072c60 2024-03-16T10:11:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8ee6ffd640 1 -- 172.21.15.17:0/1669210160 >> v1:172.21.15.174:6789/0 conn(0x7f8ee8072c60 legacy=0x7f8ee813f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8ee6ffd640 1 -- 172.21.15.17:0/1669210160 >> v1:172.21.15.17:6789/0 conn(0x7f8ee80b4da0 legacy=0x7f8ee814fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8ee67fc640 1 -- 172.21.15.17:0/1669210160 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1100925775 0 0) 0x7f8ed0003530 con 0x7f8ee80b4da0 2024-03-16T10:11:15.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.184+0000 7f8ee6ffd640 1 -- 172.21.15.17:0/1669210160 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee8149130 con 0x7f8ee80b48f0 2024-03-16T10:11:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.188+0000 7f8ee6ffd640 1 --2- 172.21.15.17:0/1669210160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ee80b48f0 0x7f8ee81487c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80097f0 tx=0x7f8ed801c2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61da06135a23271b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.188+0000 7f8ee67fc640 1 -- 172.21.15.17:0/1669210160 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80b0070 con 0x7f8ee80b48f0 2024-03-16T10:11:15.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.188+0000 7f8ee67fc640 1 -- 172.21.15.17:0/1669210160 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ed80ab400 con 0x7f8ee80b48f0 2024-03-16T10:11:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.188+0000 7f8ee67fc640 1 -- 172.21.15.17:0/1669210160 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0001a00 con 0x7f8ee80b48f0 2024-03-16T10:11:15.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.188+0000 7f8eedb60640 1 -- 172.21.15.17:0/1669210160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ee80b48f0 msgr2=0x7f8ee81487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.188+0000 7f8eedb60640 1 --2- 172.21.15.17:0/1669210160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ee80b48f0 0x7f8ee81487c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80097f0 tx=0x7f8ed801c2e0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.188+0000 7f8eedb60640 1 -- 172.21.15.17:0/1669210160 shutdown_connections 2024-03-16T10:11:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.188+0000 7f8eedb60640 1 --2- 172.21.15.17:0/1669210160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ee80b48f0 0x7f8ee81487c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.188+0000 7f8eedb60640 1 -- 172.21.15.17:0/1669210160 >> 172.21.15.17:0/1669210160 conn(0x7f8ee80b9a30 msgr2=0x7f8ee80b9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.188+0000 7f8eedb60640 1 -- 172.21.15.17:0/1669210160 shutdown_connections 2024-03-16T10:11:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.188+0000 7f8eedb60640 1 -- 172.21.15.17:0/1669210160 wait complete. 2024-03-16T10:11:15.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8eedb60640 1 Processor -- start 2024-03-16T10:11:15.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8eedb60640 1 -- start start 2024-03-16T10:11:15.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8eedb60640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ee8072c60 0x7f8ee8133f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8eedb60640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ee80b48f0 0x7f8ee812d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8eedb60640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ee80b4da0 0x7f8ee812dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8eedb60640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8153de0 con 0x7f8ee80b4da0 2024-03-16T10:11:15.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8eedb60640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ee8153c60 con 0x7f8ee8072c60 2024-03-16T10:11:15.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8eedb60640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8ee8153f60 con 0x7f8ee80b48f0 2024-03-16T10:11:15.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ee7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ee80b4da0 0x7f8ee812dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ee7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ee80b4da0 0x7f8ee812dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55674/0 (socket says 172.21.15.17:55674) 2024-03-16T10:11:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ee7fff640 1 -- 172.21.15.17:0/592388950 learned_addr learned my addr 172.21.15.17:0/592388950 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ee77fe640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ee8072c60 0x7f8ee8133f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ee6ffd640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ee80b48f0 0x7f8ee812d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ee7fff640 1 -- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ee80b48f0 msgr2=0x7f8ee812d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ee7fff640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ee80b48f0 0x7f8ee812d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ee7fff640 1 -- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ee8072c60 msgr2=0x7f8ee8133f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ee7fff640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ee8072c60 0x7f8ee8133f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ee7fff640 1 -- 172.21.15.17:0/592388950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ee8147400 con 0x7f8ee80b4da0 2024-03-16T10:11:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ee6ffd640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ee80b48f0 0x7f8ee812d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ee7fff640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ee80b4da0 0x7f8ee812dca0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0001570 tx=0x7f8ee000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ecffff640 1 -- 172.21.15.17:0/592388950 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0012400 con 0x7f8ee80b4da0 2024-03-16T10:11:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ecffff640 1 -- 172.21.15.17:0/592388950 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee00125a0 con 0x7f8ee80b4da0 2024-03-16T10:11:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8eedb60640 1 -- 172.21.15.17:0/592388950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ee8147ee0 con 0x7f8ee80b4da0 2024-03-16T10:11:15.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8ecffff640 1 -- 172.21.15.17:0/592388950 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee00129d0 con 0x7f8ee80b4da0 2024-03-16T10:11:15.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8eedb60640 1 -- 172.21.15.17:0/592388950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ee8147b70 con 0x7f8ee80b4da0 2024-03-16T10:11:15.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.192+0000 7f8eedb60640 1 -- 172.21.15.17:0/592388950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ee81480b0 con 0x7f8ee80b4da0 2024-03-16T10:11:15.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.196+0000 7f8ecffff640 1 -- 172.21.15.17:0/592388950 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8ee0012b70 con 0x7f8ee80b4da0 2024-03-16T10:11:15.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.196+0000 7f8ecffff640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8ed4067690 0x7f8ed4069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.196+0000 7f8ecffff640 1 -- 172.21.15.17:0/592388950 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8ee0086180 con 0x7f8ee80b4da0 2024-03-16T10:11:15.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.196+0000 7f8ee77fe640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8ed4067690 0x7f8ed4069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.196+0000 7f8ee77fe640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8ed4067690 0x7f8ed4069b50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0011f0 tx=0x7f8edc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:15.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.200+0000 7f8ecffff640 1 -- 172.21.15.17:0/592388950 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8ee00567a0 con 0x7f8ee80b4da0 2024-03-16T10:11:15.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.380+0000 7f8eedb60640 1 -- 172.21.15.17:0/592388950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ee8067b90 con 0x7f8ee80b4da0 2024-03-16T10:11:15.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.380+0000 7f8ecffff640 1 -- 172.21.15.17:0/592388950 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7f8ee005a840 con 0x7f8ee80b4da0 2024-03-16T10:11:15.386 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:15.386 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.384+0000 7f8eedb60640 1 -- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8ed4067690 msgr2=0x7f8ed4069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.384+0000 7f8eedb60640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8ed4067690 0x7f8ed4069b50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0011f0 tx=0x7f8edc009040 comp rx=0 tx=0).stop 2024-03-16T10:11:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.384+0000 7f8eedb60640 1 -- 172.21.15.17:0/592388950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ee80b4da0 msgr2=0x7f8ee812dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.384+0000 7f8eedb60640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ee80b4da0 0x7f8ee812dca0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0001570 tx=0x7f8ee000a430 comp rx=0 tx=0).stop 2024-03-16T10:11:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.384+0000 7f8eedb60640 1 -- 172.21.15.17:0/592388950 shutdown_connections 2024-03-16T10:11:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.384+0000 7f8eedb60640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8ed4067690 0x7f8ed4069b50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.384+0000 7f8eedb60640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ee80b4da0 0x7f8ee812dca0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.384+0000 7f8eedb60640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ee80b48f0 0x7f8ee812d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.384+0000 7f8eedb60640 1 --2- 172.21.15.17:0/592388950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ee8072c60 0x7f8ee8133f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.384+0000 7f8eedb60640 1 -- 172.21.15.17:0/592388950 >> 172.21.15.17:0/592388950 conn(0x7f8ee80b9a30 msgr2=0x7f8ee80b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.384+0000 7f8eedb60640 1 -- 172.21.15.17:0/592388950 shutdown_connections 2024-03-16T10:11:15.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.384+0000 7f8eedb60640 1 -- 172.21.15.17:0/592388950 wait complete. 2024-03-16T10:11:15.413 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:10:55.723734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:10:56.459894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:11:15.413 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:11:15.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f052bef0640 1 Processor -- start 2024-03-16T10:11:15.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f052bef0640 1 -- start start 2024-03-16T10:11:15.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f052bef0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f05240b5740 0x7f05240aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f052bef0640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f052406ede0 0x7f0524071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f052bef0640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05240b5b20 con 0x7f0524072410 2024-03-16T10:11:15.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f052bef0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f052406e950 con 0x7f05240b5740 2024-03-16T10:11:15.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f052bef0640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f05240af720 con 0x7f052406ede0 2024-03-16T10:11:15.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f0529c8c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f05240b5740 0x7f05240aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f0529c8c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f05240b5740 0x7f05240aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55690/0 (socket says 172.21.15.17:55690) 2024-03-16T10:11:15.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f0529c8c640 1 -- 172.21.15.17:0/1835300922 learned_addr learned my addr 172.21.15.17:0/1835300922 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:15.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f052948b640 1 --2- 172.21.15.17:0/1835300922 >> v2:172.21.15.174:3300/0 conn(0x7f052406ede0 0x7f0524071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f0529c8c640 1 -- 172.21.15.17:0/1835300922 >> v2:172.21.15.174:3300/0 conn(0x7f052406ede0 msgr2=0x7f0524071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:15.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f0529c8c640 1 --2- 172.21.15.17:0/1835300922 >> v2:172.21.15.174:3300/0 conn(0x7f052406ede0 0x7f0524071ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f0529c8c640 1 -- 172.21.15.17:0/1835300922 >> v1:172.21.15.174:6789/0 conn(0x7f0524072410 legacy=0x7f05241440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:15.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.532+0000 7f0529c8c640 1 -- 172.21.15.17:0/1835300922 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05240afa00 con 0x7f05240b5740 2024-03-16T10:11:15.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f0529c8c640 1 --2- 172.21.15.17:0/1835300922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05240b5740 0x7f05240aeaf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f05240b9a10 tx=0x7f052001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9438b85bdf9045 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:15.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f0528c8a640 1 -- 172.21.15.17:0/1835300922 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05200ab5f0 con 0x7f05240b5740 2024-03-16T10:11:15.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f0528c8a640 1 -- 172.21.15.17:0/1835300922 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05200b0070 con 0x7f05240b5740 2024-03-16T10:11:15.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f0528c8a640 1 -- 172.21.15.17:0/1835300922 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05200ab9b0 con 0x7f05240b5740 2024-03-16T10:11:15.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 -- 172.21.15.17:0/1835300922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05240b5740 msgr2=0x7f05240aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 --2- 172.21.15.17:0/1835300922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05240b5740 0x7f05240aeaf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f05240b9a10 tx=0x7f052001c4e0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 -- 172.21.15.17:0/1835300922 shutdown_connections 2024-03-16T10:11:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 --2- 172.21.15.17:0/1835300922 >> v2:172.21.15.174:3300/0 conn(0x7f052406ede0 0x7f0524071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 --2- 172.21.15.17:0/1835300922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05240b5740 0x7f05240aeaf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 -- 172.21.15.17:0/1835300922 >> 172.21.15.17:0/1835300922 conn(0x7f05240b65d0 msgr2=0x7f05240b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 -- 172.21.15.17:0/1835300922 shutdown_connections 2024-03-16T10:11:15.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 -- 172.21.15.17:0/1835300922 wait complete. 2024-03-16T10:11:15.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 Processor -- start 2024-03-16T10:11:15.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 -- start start 2024-03-16T10:11:15.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f052406ede0 0x7f0524146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0524072410 0x7f052413f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05240b5740 0x7f05241e0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f052406f1f0 con 0x7f052406ede0 2024-03-16T10:11:15.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f052406f370 con 0x7f0524072410 2024-03-16T10:11:15.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f05241519d0 con 0x7f05240b5740 2024-03-16T10:11:15.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f0529c8c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f052406ede0 0x7f0524146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f0529c8c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f052406ede0 0x7f0524146800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55696/0 (socket says 172.21.15.17:55696) 2024-03-16T10:11:15.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f0529c8c640 1 -- 172.21.15.17:0/2488013821 learned_addr learned my addr 172.21.15.17:0/2488013821 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:15.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052948b640 1 --2- 172.21.15.17:0/2488013821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0524072410 0x7f052413f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f0529c8c640 1 -- 172.21.15.17:0/2488013821 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05240b5740 msgr2=0x7f05241e0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f0529c8c640 1 --2- 172.21.15.17:0/2488013821 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05240b5740 0x7f05241e0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f0529c8c640 1 -- 172.21.15.17:0/2488013821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0524072410 msgr2=0x7f052413f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f0529c8c640 1 --2- 172.21.15.17:0/2488013821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0524072410 0x7f052413f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f0529c8c640 1 -- 172.21.15.17:0/2488013821 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052413fe50 con 0x7f052406ede0 2024-03-16T10:11:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f0529c8c640 1 --2- 172.21.15.17:0/2488013821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f052406ede0 0x7f0524146800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f05200ad040 tx=0x7f052001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f051a7fc640 1 -- 172.21.15.17:0/2488013821 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05200be3f0 con 0x7f052406ede0 2024-03-16T10:11:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f051a7fc640 1 -- 172.21.15.17:0/2488013821 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05200be590 con 0x7f052406ede0 2024-03-16T10:11:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 -- 172.21.15.17:0/2488013821 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05241414e0 con 0x7f052406ede0 2024-03-16T10:11:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f051a7fc640 1 -- 172.21.15.17:0/2488013821 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05200be980 con 0x7f052406ede0 2024-03-16T10:11:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.536+0000 7f052bef0640 1 -- 172.21.15.17:0/2488013821 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f052413e170 con 0x7f052406ede0 2024-03-16T10:11:15.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.540+0000 7f052bef0640 1 -- 172.21.15.17:0/2488013821 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f052413e340 con 0x7f052406ede0 2024-03-16T10:11:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.540+0000 7f051a7fc640 1 -- 172.21.15.17:0/2488013821 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f05200b43f0 con 0x7f052406ede0 2024-03-16T10:11:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.540+0000 7f051a7fc640 1 --2- 172.21.15.17:0/2488013821 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0508067660 0x7f0508069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.540+0000 7f051a7fc640 1 -- 172.21.15.17:0/2488013821 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f052011f9b0 con 0x7f052406ede0 2024-03-16T10:11:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.540+0000 7f052948b640 1 --2- 172.21.15.17:0/2488013821 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0508067660 0x7f0508069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.540+0000 7f052948b640 1 --2- 172.21.15.17:0/2488013821 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0508067660 0x7f0508069b20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0514009790 tx=0x7f0514008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:15.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.544+0000 7f051a7fc640 1 -- 172.21.15.17:0/2488013821 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f052413e340 con 0x7f052406ede0 2024-03-16T10:11:15.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.700+0000 7f052bef0640 1 -- 172.21.15.17:0/2488013821 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f052406a130 con 0x7f052406ede0 2024-03-16T10:11:15.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.700+0000 7f051a7fc640 1 -- 172.21.15.17:0/2488013821 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f05200c4060 con 0x7f052406ede0 2024-03-16T10:11:15.704 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:11:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.700+0000 7f052bef0640 1 -- 172.21.15.17:0/2488013821 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0508067660 msgr2=0x7f0508069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.700+0000 7f052bef0640 1 --2- 172.21.15.17:0/2488013821 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0508067660 0x7f0508069b20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0514009790 tx=0x7f0514008040 comp rx=0 tx=0).stop 2024-03-16T10:11:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.700+0000 7f052bef0640 1 -- 172.21.15.17:0/2488013821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f052406ede0 msgr2=0x7f0524146800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.700+0000 7f052bef0640 1 --2- 172.21.15.17:0/2488013821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f052406ede0 0x7f0524146800 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f05200ad040 tx=0x7f052001ad20 comp rx=0 tx=0).stop 2024-03-16T10:11:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.700+0000 7f052bef0640 1 -- 172.21.15.17:0/2488013821 shutdown_connections 2024-03-16T10:11:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.700+0000 7f052bef0640 1 --2- 172.21.15.17:0/2488013821 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0508067660 0x7f0508069b20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.700+0000 7f052bef0640 1 --2- 172.21.15.17:0/2488013821 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05240b5740 0x7f05241e0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.700+0000 7f052bef0640 1 --2- 172.21.15.17:0/2488013821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0524072410 0x7f052413f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.700+0000 7f052bef0640 1 --2- 172.21.15.17:0/2488013821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f052406ede0 0x7f0524146800 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.700+0000 7f052bef0640 1 -- 172.21.15.17:0/2488013821 >> 172.21.15.17:0/2488013821 conn(0x7f05240b65d0 msgr2=0x7f05241349f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:15.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.700+0000 7f052bef0640 1 -- 172.21.15.17:0/2488013821 shutdown_connections 2024-03-16T10:11:15.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.700+0000 7f052bef0640 1 -- 172.21.15.17:0/2488013821 wait complete. 2024-03-16T10:11:15.722 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:11:15.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f357b8e1640 1 Processor -- start 2024-03-16T10:11:15.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f357b8e1640 1 -- start start 2024-03-16T10:11:15.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f357b8e1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3574067220 0x7f357414ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f357b8e1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3574147070 0x7f3574151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f357b8e1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3574067600 con 0x7f357414b040 2024-03-16T10:11:15.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f357b8e1640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f357414b4f0 con 0x7f3574067220 2024-03-16T10:11:15.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f357b8e1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f35740b40f0 con 0x7f3574147070 2024-03-16T10:11:15.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f3578e7c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f357414b040 0x7f3574146960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48970/0 (socket says 172.21.15.17:48970) 2024-03-16T10:11:15.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f3578e7c640 1 -- 172.21.15.17:0/416927964 learned_addr learned my addr 172.21.15.17:0/416927964 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:15.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f357967d640 1 --2- 172.21.15.17:0/416927964 >> v2:172.21.15.174:3300/0 conn(0x7f3574067220 0x7f357414ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f3579e7e640 1 --2- 172.21.15.17:0/416927964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3574147070 0x7f3574151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 -- 172.21.15.17:0/416927964 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1559120594 0 0) 0x7f3574067600 con 0x7f357414b040 2024-03-16T10:11:15.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 -- 172.21.15.17:0/416927964 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3560003530 con 0x7f357414b040 2024-03-16T10:11:15.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 -- 172.21.15.17:0/416927964 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1244563058 0 0) 0x7f3560003530 con 0x7f357414b040 2024-03-16T10:11:15.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 -- 172.21.15.17:0/416927964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3574147070 msgr2=0x7f3574151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:15.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 --2- 172.21.15.17:0/416927964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3574147070 0x7f3574151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 -- 172.21.15.17:0/416927964 >> v2:172.21.15.174:3300/0 conn(0x7f3574067220 msgr2=0x7f357414ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:15.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 --2- 172.21.15.17:0/416927964 >> v2:172.21.15.174:3300/0 conn(0x7f3574067220 0x7f357414ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 -- 172.21.15.17:0/416927964 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35740b43d0 con 0x7f357414b040 2024-03-16T10:11:15.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 -- 172.21.15.17:0/416927964 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f3564001680 con 0x7f357414b040 2024-03-16T10:11:15.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 -- 172.21.15.17:0/416927964 >> v1:172.21.15.17:6789/0 conn(0x7f357414b040 legacy=0x7f3574146960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 --2- 172.21.15.17:0/416927964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3574147070 0x7f35600039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 --2- 172.21.15.17:0/416927964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3560003f60 0x7f3560004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 --2- 172.21.15.17:0/416927964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3560004d50 0x7f35600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 -- 172.21.15.17:0/416927964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f357414b4f0 con 0x7f3560003f60 2024-03-16T10:11:15.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 -- 172.21.15.17:0/416927964 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35740b40f0 con 0x7f3560004d50 2024-03-16T10:11:15.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f356bfff640 1 -- 172.21.15.17:0/416927964 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f356000ba50 con 0x7f3574147070 2024-03-16T10:11:15.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f3578e7c640 1 --2- 172.21.15.17:0/416927964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3560003f60 0x7f3560004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f357967d640 1 --2- 172.21.15.17:0/416927964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3574147070 0x7f35600039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f3579e7e640 1 --2- 172.21.15.17:0/416927964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3560004d50 0x7f35600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f3578e7c640 1 -- 172.21.15.17:0/416927964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3574147070 msgr2=0x7f35600039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:15.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f3578e7c640 1 --2- 172.21.15.17:0/416927964 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3574147070 0x7f35600039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f3578e7c640 1 -- 172.21.15.17:0/416927964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3560004d50 msgr2=0x7f35600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:15.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f3578e7c640 1 --2- 172.21.15.17:0/416927964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3560004d50 0x7f35600051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f3578e7c640 1 -- 172.21.15.17:0/416927964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3560003530 con 0x7f3560003f60 2024-03-16T10:11:15.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.844+0000 7f3579e7e640 1 --2- 172.21.15.17:0/416927964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3560004d50 0x7f35600051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:15.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3578e7c640 1 --2- 172.21.15.17:0/416927964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3560003f60 0x7f3560004810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f356400c240 tx=0x7f356401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcc031b4e910b26 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:15.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f356bfff640 1 -- 172.21.15.17:0/416927964 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564007d40 con 0x7f3560003f60 2024-03-16T10:11:15.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f356bfff640 1 -- 172.21.15.17:0/416927964 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35640a9020 con 0x7f3560003f60 2024-03-16T10:11:15.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f356bfff640 1 -- 172.21.15.17:0/416927964 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564005ae0 con 0x7f3560003f60 2024-03-16T10:11:15.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 -- 172.21.15.17:0/416927964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3560003f60 msgr2=0x7f3560004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 --2- 172.21.15.17:0/416927964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3560003f60 0x7f3560004810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f356400c240 tx=0x7f356401cca0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3579e7e640 1 -- 172.21.15.17:0/416927964 reap_dead start 2024-03-16T10:11:15.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 -- 172.21.15.17:0/416927964 shutdown_connections 2024-03-16T10:11:15.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 -- 172.21.15.17:0/416927964 >> 172.21.15.17:0/416927964 conn(0x7f357406bd70 msgr2=0x7f35740b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:15.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 -- 172.21.15.17:0/416927964 shutdown_connections 2024-03-16T10:11:15.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 -- 172.21.15.17:0/416927964 wait complete. 2024-03-16T10:11:15.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 Processor -- start 2024-03-16T10:11:15.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 -- start start 2024-03-16T10:11:15.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f357409d2f0 0x7f35741e3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f357409dbe0 0x7f35741e3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f357409d6d0 0x7f3574140440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f35740b3ca0 con 0x7f357409dbe0 2024-03-16T10:11:15.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35740b5ad0 con 0x7f357409d6d0 2024-03-16T10:11:15.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f35740b3e20 con 0x7f357409d2f0 2024-03-16T10:11:15.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3578e7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f357409dbe0 0x7f35741e3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3578e7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f357409dbe0 0x7f35741e3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55704/0 (socket says 172.21.15.17:55704) 2024-03-16T10:11:15.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3578e7c640 1 -- 172.21.15.17:0/78828296 learned_addr learned my addr 172.21.15.17:0/78828296 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:15.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357967d640 1 --2- 172.21.15.17:0/78828296 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f357409d2f0 0x7f35741e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3579e7e640 1 --2- 172.21.15.17:0/78828296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f357409d6d0 0x7f3574140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3578e7c640 1 -- 172.21.15.17:0/78828296 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f357409d2f0 msgr2=0x7f35741e3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3578e7c640 1 --2- 172.21.15.17:0/78828296 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f357409d2f0 0x7f35741e3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3578e7c640 1 -- 172.21.15.17:0/78828296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f357409d6d0 msgr2=0x7f3574140440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3578e7c640 1 --2- 172.21.15.17:0/78828296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f357409d6d0 0x7f3574140440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3578e7c640 1 -- 172.21.15.17:0/78828296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35741e3200 con 0x7f357409dbe0 2024-03-16T10:11:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3578e7c640 1 --2- 172.21.15.17:0/78828296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f357409dbe0 0x7f35741e3c40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f356401c710 tx=0x7f3560005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3569ffb640 1 -- 172.21.15.17:0/78828296 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564001c80 con 0x7f357409dbe0 2024-03-16T10:11:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3569ffb640 1 -- 172.21.15.17:0/78828296 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35640a5a90 con 0x7f357409dbe0 2024-03-16T10:11:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 -- 172.21.15.17:0/78828296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3574070510 con 0x7f357409dbe0 2024-03-16T10:11:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f3569ffb640 1 -- 172.21.15.17:0/78828296 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35640a5e40 con 0x7f357409dbe0 2024-03-16T10:11:15.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.848+0000 7f357b8e1640 1 -- 172.21.15.17:0/78828296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3574071710 con 0x7f357409dbe0 2024-03-16T10:11:15.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.852+0000 7f357b8e1640 1 -- 172.21.15.17:0/78828296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f357406a130 con 0x7f357409dbe0 2024-03-16T10:11:15.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.852+0000 7f3569ffb640 1 -- 172.21.15.17:0/78828296 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f35640047f0 con 0x7f357409dbe0 2024-03-16T10:11:15.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.852+0000 7f3569ffb640 1 --2- 172.21.15.17:0/78828296 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3558067660 0x7f3558069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:15.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.852+0000 7f3569ffb640 1 -- 172.21.15.17:0/78828296 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f356411ee00 con 0x7f357409dbe0 2024-03-16T10:11:15.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.852+0000 7f357967d640 1 --2- 172.21.15.17:0/78828296 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3558067660 0x7f3558069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:15.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.852+0000 7f357967d640 1 --2- 172.21.15.17:0/78828296 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3558067660 0x7f3558069b20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3570009a30 tx=0x7f3570008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:15.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:15.856+0000 7f3569ffb640 1 -- 172.21.15.17:0/78828296 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f35640169b0 con 0x7f357409dbe0 2024-03-16T10:11:16.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.012+0000 7f357b8e1640 1 -- 172.21.15.17:0/78828296 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3574145770 con 0x7f357409dbe0 2024-03-16T10:11:16.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.012+0000 7f3569ffb640 1 -- 172.21.15.17:0/78828296 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f35640efb10 con 0x7f357409dbe0 2024-03-16T10:11:16.019 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-16T10:11:16.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.016+0000 7f357b8e1640 1 -- 172.21.15.17:0/78828296 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3558067660 msgr2=0x7f3558069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.016+0000 7f357b8e1640 1 --2- 172.21.15.17:0/78828296 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3558067660 0x7f3558069b20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3570009a30 tx=0x7f3570008040 comp rx=0 tx=0).stop 2024-03-16T10:11:16.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.016+0000 7f357b8e1640 1 -- 172.21.15.17:0/78828296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f357409dbe0 msgr2=0x7f35741e3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.016+0000 7f357b8e1640 1 --2- 172.21.15.17:0/78828296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f357409dbe0 0x7f35741e3c40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f356401c710 tx=0x7f3560005840 comp rx=0 tx=0).stop 2024-03-16T10:11:16.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.016+0000 7f357b8e1640 1 -- 172.21.15.17:0/78828296 shutdown_connections 2024-03-16T10:11:16.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.016+0000 7f357b8e1640 1 --2- 172.21.15.17:0/78828296 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3558067660 0x7f3558069b20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.016+0000 7f357b8e1640 1 --2- 172.21.15.17:0/78828296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f357409d6d0 0x7f3574140440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.016+0000 7f357b8e1640 1 --2- 172.21.15.17:0/78828296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f357409dbe0 0x7f35741e3c40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.016+0000 7f357b8e1640 1 --2- 172.21.15.17:0/78828296 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f357409d2f0 0x7f35741e3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.016+0000 7f357b8e1640 1 -- 172.21.15.17:0/78828296 >> 172.21.15.17:0/78828296 conn(0x7f357406bd70 msgr2=0x7f3574147590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:16.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.016+0000 7f357b8e1640 1 -- 172.21.15.17:0/78828296 shutdown_connections 2024-03-16T10:11:16.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.016+0000 7f357b8e1640 1 -- 172.21.15.17:0/78828296 wait complete. 2024-03-16T10:11:16.036 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:11:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66e23b5640 1 Processor -- start 2024-03-16T10:11:16.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66e23b5640 1 -- start start 2024-03-16T10:11:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66e23b5640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f66dc073070 0x7f66dc0b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66e23b5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66dc0b6fd0 con 0x7f66dc0b5620 2024-03-16T10:11:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66e23b5640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66dc0b64f0 con 0x7f66dc0b7770 2024-03-16T10:11:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66e23b5640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f66dc0b67d0 con 0x7f66dc073070 2024-03-16T10:11:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66e0952640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f66dc0b5620 0x7f66dc14faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48974/0 (socket says 172.21.15.17:48974) 2024-03-16T10:11:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66e0952640 1 -- 172.21.15.17:0/3936014301 learned_addr learned my addr 172.21.15.17:0/3936014301 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66db7fe640 1 --2- 172.21.15.17:0/3936014301 >> v2:172.21.15.174:3300/0 conn(0x7f66dc073070 0x7f66dc0b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 -- 172.21.15.17:0/3936014301 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2088354730 0 0) 0x7f66dc0b6fd0 con 0x7f66dc0b5620 2024-03-16T10:11:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 -- 172.21.15.17:0/3936014301 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66c4003530 con 0x7f66dc0b5620 2024-03-16T10:11:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 -- 172.21.15.17:0/3936014301 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3887740038 0 0) 0x7f66c4003530 con 0x7f66dc0b5620 2024-03-16T10:11:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 -- 172.21.15.17:0/3936014301 >> v2:172.21.15.174:3300/0 conn(0x7f66dc073070 msgr2=0x7f66dc0b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 --2- 172.21.15.17:0/3936014301 >> v2:172.21.15.174:3300/0 conn(0x7f66dc073070 0x7f66dc0b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 -- 172.21.15.17:0/3936014301 >> v1:172.21.15.174:6789/0 conn(0x7f66dc0b7770 legacy=0x7f66dc145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 -- 172.21.15.17:0/3936014301 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66dc0b6ab0 con 0x7f66dc0b5620 2024-03-16T10:11:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 -- 172.21.15.17:0/3936014301 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f66d4001a00 con 0x7f66dc0b5620 2024-03-16T10:11:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 -- 172.21.15.17:0/3936014301 >> v1:172.21.15.17:6789/0 conn(0x7f66dc0b5620 legacy=0x7f66dc14faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 --2- 172.21.15.17:0/3936014301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66c4003a10 0x7f66c4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 --2- 172.21.15.17:0/3936014301 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66c4004810 0x7f66c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 --2- 172.21.15.17:0/3936014301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c40052f0 0x7f66c400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 -- 172.21.15.17:0/3936014301 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc0b67d0 con 0x7f66c40052f0 2024-03-16T10:11:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 -- 172.21.15.17:0/3936014301 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66c400c1f0 con 0x7f66c4003a10 2024-03-16T10:11:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66daffd640 1 -- 172.21.15.17:0/3936014301 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f66c400c3f0 con 0x7f66c4004810 2024-03-16T10:11:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66e0952640 1 --2- 172.21.15.17:0/3936014301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c40052f0 0x7f66c400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66dbfff640 1 --2- 172.21.15.17:0/3936014301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66c4003a10 0x7f66c4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66db7fe640 1 --2- 172.21.15.17:0/3936014301 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66c4004810 0x7f66c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66e0952640 1 -- 172.21.15.17:0/3936014301 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66c4004810 msgr2=0x7f66c4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66e0952640 1 --2- 172.21.15.17:0/3936014301 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66c4004810 0x7f66c4004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66e0952640 1 -- 172.21.15.17:0/3936014301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66c4003a10 msgr2=0x7f66c4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66e0952640 1 --2- 172.21.15.17:0/3936014301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66c4003a10 0x7f66c4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.152+0000 7f66e0952640 1 -- 172.21.15.17:0/3936014301 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66c4003530 con 0x7f66c40052f0 2024-03-16T10:11:16.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e0952640 1 -- 172.21.15.17:0/3936014301 reap_dead start 2024-03-16T10:11:16.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e0952640 1 --2- 172.21.15.17:0/3936014301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c40052f0 0x7f66c400b9d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f66d400a770 tx=0x7f66d401d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfea2eeaefdd2ab server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66daffd640 1 -- 172.21.15.17:0/3936014301 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d400a390 con 0x7f66c40052f0 2024-03-16T10:11:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66daffd640 1 -- 172.21.15.17:0/3936014301 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d40ac640 con 0x7f66c40052f0 2024-03-16T10:11:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66daffd640 1 -- 172.21.15.17:0/3936014301 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d40aca30 con 0x7f66c40052f0 2024-03-16T10:11:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 -- 172.21.15.17:0/3936014301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c40052f0 msgr2=0x7f66c400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 --2- 172.21.15.17:0/3936014301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c40052f0 0x7f66c400b9d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f66d400a770 tx=0x7f66d401d2a0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 -- 172.21.15.17:0/3936014301 shutdown_connections 2024-03-16T10:11:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 --2- 172.21.15.17:0/3936014301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66c40052f0 0x7f66c400b9d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 -- 172.21.15.17:0/3936014301 >> 172.21.15.17:0/3936014301 conn(0x7f66dc06b290 msgr2=0x7f66dc06b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 -- 172.21.15.17:0/3936014301 shutdown_connections 2024-03-16T10:11:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 -- 172.21.15.17:0/3936014301 wait complete. 2024-03-16T10:11:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 Processor -- start 2024-03-16T10:11:16.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 -- start start 2024-03-16T10:11:16.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66c40052f0 0x7f66dc13f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66dc0aa820 0x7f66dc1e11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66dc1e1720 0x7f66dc0a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc0b4440 con 0x7f66dc0aa820 2024-03-16T10:11:16.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66dc0b42c0 con 0x7f66dc1e1720 2024-03-16T10:11:16.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66e23b5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f66dc0b45c0 con 0x7f66c40052f0 2024-03-16T10:11:16.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66db7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66dc0aa820 0x7f66dc1e11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66db7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66dc0aa820 0x7f66dc1e11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55728/0 (socket says 172.21.15.17:55728) 2024-03-16T10:11:16.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.156+0000 7f66db7fe640 1 -- 172.21.15.17:0/3254104038 learned_addr learned my addr 172.21.15.17:0/3254104038 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:16.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66dbfff640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66c40052f0 0x7f66dc13f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66e0952640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66dc1e1720 0x7f66dc0a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66db7fe640 1 -- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66c40052f0 msgr2=0x7f66dc13f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66db7fe640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66c40052f0 0x7f66dc13f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66db7fe640 1 -- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66dc1e1720 msgr2=0x7f66dc0a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66db7fe640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66dc1e1720 0x7f66dc0a94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66db7fe640 1 -- 172.21.15.17:0/3254104038 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66dc0abe50 con 0x7f66dc0aa820 2024-03-16T10:11:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66e0952640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66dc1e1720 0x7f66dc0a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66db7fe640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66dc0aa820 0x7f66dc1e11e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66d0010890 tx=0x7f66d0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66d8ff9640 1 -- 172.21.15.17:0/3254104038 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d000f040 con 0x7f66dc0aa820 2024-03-16T10:11:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66d8ff9640 1 -- 172.21.15.17:0/3254104038 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d000eb60 con 0x7f66dc0aa820 2024-03-16T10:11:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66e23b5640 1 -- 172.21.15.17:0/3254104038 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66dc0ac130 con 0x7f66dc0aa820 2024-03-16T10:11:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66d8ff9640 1 -- 172.21.15.17:0/3254104038 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d001c400 con 0x7f66dc0aa820 2024-03-16T10:11:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66e23b5640 1 -- 172.21.15.17:0/3254104038 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66dc0bb1e0 con 0x7f66dc0aa820 2024-03-16T10:11:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66e23b5640 1 -- 172.21.15.17:0/3254104038 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66dc06a130 con 0x7f66dc0aa820 2024-03-16T10:11:16.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66d8ff9640 1 -- 172.21.15.17:0/3254104038 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f66d001c5a0 con 0x7f66dc0aa820 2024-03-16T10:11:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66d8ff9640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f66c8067660 0x7f66c8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66d8ff9640 1 -- 172.21.15.17:0/3254104038 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f66d0085ec0 con 0x7f66dc0aa820 2024-03-16T10:11:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66dbfff640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f66c8067660 0x7f66c8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.160+0000 7f66dbfff640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f66c8067660 0x7f66c8069b20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f66cc005410 tx=0x7f66cc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:16.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.164+0000 7f66d8ff9640 1 -- 172.21.15.17:0/3254104038 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f66d0018030 con 0x7f66dc0aa820 2024-03-16T10:11:16.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.424+0000 7f66e23b5640 1 -- 172.21.15.17:0/3254104038 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f66dc0b7e10 con 0x7f66dc0aa820 2024-03-16T10:11:16.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.428+0000 7f66d8ff9640 1 -- 172.21.15.17:0/3254104038 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f66d0056bd0 con 0x7f66dc0aa820 2024-03-16T10:11:16.433 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-16T10:11:16.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.428+0000 7f66e23b5640 1 -- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f66c8067660 msgr2=0x7f66c8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.428+0000 7f66e23b5640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f66c8067660 0x7f66c8069b20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f66cc005410 tx=0x7f66cc009040 comp rx=0 tx=0).stop 2024-03-16T10:11:16.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.428+0000 7f66e23b5640 1 -- 172.21.15.17:0/3254104038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66dc0aa820 msgr2=0x7f66dc1e11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.428+0000 7f66e23b5640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66dc0aa820 0x7f66dc1e11e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66d0010890 tx=0x7f66d0010d50 comp rx=0 tx=0).stop 2024-03-16T10:11:16.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.428+0000 7f66e23b5640 1 -- 172.21.15.17:0/3254104038 shutdown_connections 2024-03-16T10:11:16.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.428+0000 7f66e23b5640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f66c8067660 0x7f66c8069b20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.428+0000 7f66e23b5640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66dc1e1720 0x7f66dc0a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.428+0000 7f66e23b5640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66dc0aa820 0x7f66dc1e11e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.428+0000 7f66e23b5640 1 --2- 172.21.15.17:0/3254104038 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f66c40052f0 0x7f66dc13f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.428+0000 7f66e23b5640 1 -- 172.21.15.17:0/3254104038 >> 172.21.15.17:0/3254104038 conn(0x7f66dc06b290 msgr2=0x7f66dc06b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:16.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.428+0000 7f66e23b5640 1 -- 172.21.15.17:0/3254104038 shutdown_connections 2024-03-16T10:11:16.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.428+0000 7f66e23b5640 1 -- 172.21.15.17:0/3254104038 wait complete. 2024-03-16T10:11:16.449 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:16.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6edeb5640 1 Processor -- start 2024-03-16T10:11:16.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6edeb5640 1 -- start start 2024-03-16T10:11:16.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6edeb5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6e8066fc0 0x7fb6e80b4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6edeb5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6e81484d0 con 0x7fb6e80673a0 2024-03-16T10:11:16.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6edeb5640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6e80b3eb0 con 0x7fb6e8141530 2024-03-16T10:11:16.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6edeb5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e80b4150 con 0x7fb6e8066fc0 2024-03-16T10:11:16.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e6ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb6e80673a0 0x7fb6e8147a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48990/0 (socket says 172.21.15.17:48990) 2024-03-16T10:11:16.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e6ffd640 1 -- 172.21.15.17:0/1989586765 learned_addr learned my addr 172.21.15.17:0/1989586765 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:16.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e77fe640 1 --2- 172.21.15.17:0/1989586765 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6e8066fc0 0x7fb6e80b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 -- 172.21.15.17:0/1989586765 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 337037015 0 0) 0x7fb6e81484d0 con 0x7fb6e80673a0 2024-03-16T10:11:16.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 -- 172.21.15.17:0/1989586765 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6d0003530 con 0x7fb6e80673a0 2024-03-16T10:11:16.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 -- 172.21.15.17:0/1989586765 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3405714123 0 0) 0x7fb6d0003530 con 0x7fb6e80673a0 2024-03-16T10:11:16.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 -- 172.21.15.17:0/1989586765 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6e8066fc0 msgr2=0x7fb6e80b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 --2- 172.21.15.17:0/1989586765 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6e8066fc0 0x7fb6e80b4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 -- 172.21.15.17:0/1989586765 >> v1:172.21.15.174:6789/0 conn(0x7fb6e8141530 legacy=0x7fb6e8151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 -- 172.21.15.17:0/1989586765 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e809bf30 con 0x7fb6e80673a0 2024-03-16T10:11:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 -- 172.21.15.17:0/1989586765 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fb6d8001680 con 0x7fb6e80673a0 2024-03-16T10:11:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 -- 172.21.15.17:0/1989586765 >> v1:172.21.15.17:6789/0 conn(0x7fb6e80673a0 legacy=0x7fb6e8147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 --2- 172.21.15.17:0/1989586765 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6e8066fc0 0x7fb6d00039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 --2- 172.21.15.17:0/1989586765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6d0003f70 0x7fb6d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 --2- 172.21.15.17:0/1989586765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6d0004d50 0x7fb6d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 -- 172.21.15.17:0/1989586765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e80b4150 con 0x7fb6d0003f70 2024-03-16T10:11:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 -- 172.21.15.17:0/1989586765 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d000bc10 con 0x7fb6d0004d50 2024-03-16T10:11:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e67fc640 1 -- 172.21.15.17:0/1989586765 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d000be10 con 0x7fb6e8066fc0 2024-03-16T10:11:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e6ffd640 1 --2- 172.21.15.17:0/1989586765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6d0003f70 0x7fb6d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e77fe640 1 --2- 172.21.15.17:0/1989586765 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6e8066fc0 0x7fb6d00039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e7fff640 1 --2- 172.21.15.17:0/1989586765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6d0004d50 0x7fb6d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e6ffd640 1 -- 172.21.15.17:0/1989586765 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6e8066fc0 msgr2=0x7fb6d00039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:16.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e6ffd640 1 --2- 172.21.15.17:0/1989586765 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6e8066fc0 0x7fb6d00039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e6ffd640 1 -- 172.21.15.17:0/1989586765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6d0004d50 msgr2=0x7fb6d00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:16.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e6ffd640 1 --2- 172.21.15.17:0/1989586765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6d0004d50 0x7fb6d00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.556+0000 7fb6e6ffd640 1 -- 172.21.15.17:0/1989586765 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d0003530 con 0x7fb6d0003f70 2024-03-16T10:11:16.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.560+0000 7fb6e6ffd640 1 --2- 172.21.15.17:0/1989586765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6d0003f70 0x7fb6d0004810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800c210 tx=0x7fb6d801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45dc5eaa7625eb0b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:16.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.560+0000 7fb6e67fc640 1 -- 172.21.15.17:0/1989586765 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8007d40 con 0x7fb6d0003f70 2024-03-16T10:11:16.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.560+0000 7fb6e67fc640 1 -- 172.21.15.17:0/1989586765 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6d80a9070 con 0x7fb6d0003f70 2024-03-16T10:11:16.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.560+0000 7fb6e67fc640 1 -- 172.21.15.17:0/1989586765 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8005d80 con 0x7fb6d0003f70 2024-03-16T10:11:16.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.560+0000 7fb6edeb5640 1 -- 172.21.15.17:0/1989586765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6d0003f70 msgr2=0x7fb6d0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.560+0000 7fb6edeb5640 1 --2- 172.21.15.17:0/1989586765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6d0003f70 0x7fb6d0004810 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800c210 tx=0x7fb6d801cca0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.560+0000 7fb6e7fff640 1 -- 172.21.15.17:0/1989586765 reap_dead start 2024-03-16T10:11:16.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.560+0000 7fb6edeb5640 1 -- 172.21.15.17:0/1989586765 shutdown_connections 2024-03-16T10:11:16.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.560+0000 7fb6edeb5640 1 -- 172.21.15.17:0/1989586765 >> 172.21.15.17:0/1989586765 conn(0x7fb6e809b130 msgr2=0x7fb6e809b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:16.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.560+0000 7fb6edeb5640 1 -- 172.21.15.17:0/1989586765 shutdown_connections 2024-03-16T10:11:16.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.560+0000 7fb6edeb5640 1 -- 172.21.15.17:0/1989586765 wait complete. 2024-03-16T10:11:16.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6edeb5640 1 Processor -- start 2024-03-16T10:11:16.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6edeb5640 1 -- start start 2024-03-16T10:11:16.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6edeb5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6e814a620 0x7fb6e81e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6edeb5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6e80b33c0 0x7fb6e809cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6edeb5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e813c930 0x7fb6e813cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6edeb5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e81538b0 con 0x7fb6e813c930 2024-03-16T10:11:16.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6edeb5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6e8153730 con 0x7fb6e80b33c0 2024-03-16T10:11:16.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6edeb5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb6e8153a30 con 0x7fb6e814a620 2024-03-16T10:11:16.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e813c930 0x7fb6e813cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e813c930 0x7fb6e813cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55734/0 (socket says 172.21.15.17:55734) 2024-03-16T10:11:16.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6e7fff640 1 -- 172.21.15.17:0/100385494 learned_addr learned my addr 172.21.15.17:0/100385494 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:16.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6e77fe640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6e814a620 0x7fb6e81e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6e6ffd640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6e80b33c0 0x7fb6e809cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6e7fff640 1 -- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6e814a620 msgr2=0x7fb6e81e3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6e7fff640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6e814a620 0x7fb6e81e3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6e7fff640 1 -- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6e80b33c0 msgr2=0x7fb6e809cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6e7fff640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6e80b33c0 0x7fb6e809cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6e7fff640 1 -- 172.21.15.17:0/100385494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6e814b2c0 con 0x7fb6e813c930 2024-03-16T10:11:16.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6e6ffd640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6e80b33c0 0x7fb6e809cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:16.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6e7fff640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e813c930 0x7fb6e813cdc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6e806fb20 tx=0x7fb6e000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:16.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6cffff640 1 -- 172.21.15.17:0/100385494 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e000ee90 con 0x7fb6e813c930 2024-03-16T10:11:16.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6cffff640 1 -- 172.21.15.17:0/100385494 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e0018070 con 0x7fb6e813c930 2024-03-16T10:11:16.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6cffff640 1 -- 172.21.15.17:0/100385494 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0003bc0 con 0x7fb6e813c930 2024-03-16T10:11:16.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6edeb5640 1 -- 172.21.15.17:0/100385494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6e81e4230 con 0x7fb6e813c930 2024-03-16T10:11:16.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6edeb5640 1 -- 172.21.15.17:0/100385494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6e814aa00 con 0x7fb6e813c930 2024-03-16T10:11:16.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6edeb5640 1 -- 172.21.15.17:0/100385494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6e806a130 con 0x7fb6e813c930 2024-03-16T10:11:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.564+0000 7fb6cffff640 1 -- 172.21.15.17:0/100385494 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fb6e00050d0 con 0x7fb6e813c930 2024-03-16T10:11:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.568+0000 7fb6cffff640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb6d4067800 0x7fb6d4069cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.568+0000 7fb6cffff640 1 -- 172.21.15.17:0/100385494 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb6e0085370 con 0x7fb6e813c930 2024-03-16T10:11:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.568+0000 7fb6e77fe640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb6d4067800 0x7fb6d4069cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.568+0000 7fb6e77fe640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb6d4067800 0x7fb6d4069cc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6e814af60 tx=0x7fb6dc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:16.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.576+0000 7fb6cffff640 1 -- 172.21.15.17:0/100385494 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb6e0056080 con 0x7fb6e813c930 2024-03-16T10:11:16.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.736+0000 7fb6edeb5640 1 -- 172.21.15.17:0/100385494 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb6e81451b0 con 0x7fb6e813c930 2024-03-16T10:11:16.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.736+0000 7fb6cffff640 1 -- 172.21.15.17:0/100385494 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7fb6e0055980 con 0x7fb6e813c930 2024-03-16T10:11:16.742 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:16.742 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:16.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.740+0000 7fb6edeb5640 1 -- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb6d4067800 msgr2=0x7fb6d4069cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.740+0000 7fb6edeb5640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb6d4067800 0x7fb6d4069cc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6e814af60 tx=0x7fb6dc008040 comp rx=0 tx=0).stop 2024-03-16T10:11:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.740+0000 7fb6edeb5640 1 -- 172.21.15.17:0/100385494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e813c930 msgr2=0x7fb6e813cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.740+0000 7fb6edeb5640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e813c930 0x7fb6e813cdc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb6e806fb20 tx=0x7fb6e000eb50 comp rx=0 tx=0).stop 2024-03-16T10:11:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.740+0000 7fb6edeb5640 1 -- 172.21.15.17:0/100385494 shutdown_connections 2024-03-16T10:11:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.740+0000 7fb6edeb5640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb6d4067800 0x7fb6d4069cc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.740+0000 7fb6edeb5640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb6e813c930 0x7fb6e813cdc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.740+0000 7fb6edeb5640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb6e80b33c0 0x7fb6e809cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.740+0000 7fb6edeb5640 1 --2- 172.21.15.17:0/100385494 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb6e814a620 0x7fb6e81e3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.740+0000 7fb6edeb5640 1 -- 172.21.15.17:0/100385494 >> 172.21.15.17:0/100385494 conn(0x7fb6e809b130 msgr2=0x7fb6e8146380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.740+0000 7fb6edeb5640 1 -- 172.21.15.17:0/100385494 shutdown_connections 2024-03-16T10:11:16.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.740+0000 7fb6edeb5640 1 -- 172.21.15.17:0/100385494 wait complete. 2024-03-16T10:11:16.759 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:10:55.723734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:10:56.459894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:11:16.760 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:11:16.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a2859a640 1 Processor -- start 2024-03-16T10:11:16.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a2859a640 1 -- start start 2024-03-16T10:11:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a2859a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9a20067220 0x7f9a200b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a2859a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a20067600 con 0x7f9a201413d0 2024-03-16T10:11:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a2859a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a200b5470 con 0x7f9a20147940 2024-03-16T10:11:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a2859a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f9a200b5ac0 con 0x7f9a20067220 2024-03-16T10:11:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a26b37640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9a201413d0 0x7f9a20141840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49002/0 (socket says 172.21.15.17:49002) 2024-03-16T10:11:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a26b37640 1 -- 172.21.15.17:0/2889205570 learned_addr learned my addr 172.21.15.17:0/2889205570 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a26336640 1 --2- 172.21.15.17:0/2889205570 >> v2:172.21.15.174:3300/0 conn(0x7f9a20067220 0x7f9a200b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 -- 172.21.15.17:0/2889205570 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 311376535 0 0) 0x7f9a20067600 con 0x7f9a201413d0 2024-03-16T10:11:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 -- 172.21.15.17:0/2889205570 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a0c003530 con 0x7f9a201413d0 2024-03-16T10:11:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 -- 172.21.15.17:0/2889205570 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 460121508 0 0) 0x7f9a0c003530 con 0x7f9a201413d0 2024-03-16T10:11:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 -- 172.21.15.17:0/2889205570 >> v2:172.21.15.174:3300/0 conn(0x7f9a20067220 msgr2=0x7f9a200b4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 --2- 172.21.15.17:0/2889205570 >> v2:172.21.15.174:3300/0 conn(0x7f9a20067220 0x7f9a200b4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 -- 172.21.15.17:0/2889205570 >> v1:172.21.15.174:6789/0 conn(0x7f9a20147940 legacy=0x7f9a20147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 -- 172.21.15.17:0/2889205570 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a200b5750 con 0x7f9a201413d0 2024-03-16T10:11:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 -- 172.21.15.17:0/2889205570 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f9a1c001a00 con 0x7f9a201413d0 2024-03-16T10:11:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 -- 172.21.15.17:0/2889205570 >> v1:172.21.15.17:6789/0 conn(0x7f9a201413d0 legacy=0x7f9a20141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 --2- 172.21.15.17:0/2889205570 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a0c0039c0 0x7f9a0c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 --2- 172.21.15.17:0/2889205570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a0c004810 0x7f9a0c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 --2- 172.21.15.17:0/2889205570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a0c005220 0x7f9a0c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 -- 172.21.15.17:0/2889205570 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9a200b5ac0 con 0x7f9a0c005220 2024-03-16T10:11:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 -- 172.21.15.17:0/2889205570 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c00c150 con 0x7f9a0c004810 2024-03-16T10:11:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25334640 1 -- 172.21.15.17:0/2889205570 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0c00c350 con 0x7f9a0c0039c0 2024-03-16T10:11:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a26b37640 1 --2- 172.21.15.17:0/2889205570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a0c005220 0x7f9a0c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a26336640 1 --2- 172.21.15.17:0/2889205570 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a0c0039c0 0x7f9a0c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25b35640 1 --2- 172.21.15.17:0/2889205570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a0c004810 0x7f9a0c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a26b37640 1 -- 172.21.15.17:0/2889205570 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a0c0039c0 msgr2=0x7f9a0c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a26b37640 1 --2- 172.21.15.17:0/2889205570 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a0c0039c0 0x7f9a0c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a26b37640 1 -- 172.21.15.17:0/2889205570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a0c004810 msgr2=0x7f9a0c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a26b37640 1 --2- 172.21.15.17:0/2889205570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a0c004810 0x7f9a0c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a26b37640 1 -- 172.21.15.17:0/2889205570 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0c003530 con 0x7f9a0c005220 2024-03-16T10:11:16.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.872+0000 7f9a25b35640 1 --2- 172.21.15.17:0/2889205570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a0c004810 0x7f9a0c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.876+0000 7f9a26b37640 1 -- 172.21.15.17:0/2889205570 reap_dead start 2024-03-16T10:11:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.876+0000 7f9a26b37640 1 --2- 172.21.15.17:0/2889205570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a0c005220 0x7f9a0c00b9d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00a440 tx=0x7f9a1c021d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ad75951f51ff83 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.876+0000 7f9a25334640 1 -- 172.21.15.17:0/2889205570 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00a480 con 0x7f9a0c005220 2024-03-16T10:11:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.876+0000 7f9a25334640 1 -- 172.21.15.17:0/2889205570 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a1c0af070 con 0x7f9a0c005220 2024-03-16T10:11:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.876+0000 7f9a25334640 1 -- 172.21.15.17:0/2889205570 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0abb70 con 0x7f9a0c005220 2024-03-16T10:11:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.876+0000 7f9a2859a640 1 -- 172.21.15.17:0/2889205570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a0c005220 msgr2=0x7f9a0c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.876+0000 7f9a2859a640 1 --2- 172.21.15.17:0/2889205570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a0c005220 0x7f9a0c00b9d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00a440 tx=0x7f9a1c021d10 comp rx=0 tx=0).stop 2024-03-16T10:11:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.876+0000 7f9a2859a640 1 -- 172.21.15.17:0/2889205570 shutdown_connections 2024-03-16T10:11:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.876+0000 7f9a2859a640 1 --2- 172.21.15.17:0/2889205570 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a0c005220 0x7f9a0c00b9d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.876+0000 7f9a2859a640 1 -- 172.21.15.17:0/2889205570 >> 172.21.15.17:0/2889205570 conn(0x7f9a2006bd70 msgr2=0x7f9a200b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:16.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.876+0000 7f9a2859a640 1 -- 172.21.15.17:0/2889205570 shutdown_connections 2024-03-16T10:11:16.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.876+0000 7f9a2859a640 1 -- 172.21.15.17:0/2889205570 wait complete. 2024-03-16T10:11:16.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a2859a640 1 Processor -- start 2024-03-16T10:11:16.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a2859a640 1 -- start start 2024-03-16T10:11:16.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a2859a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a0c005220 0x7f9a2009cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a2859a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a200b0f30 0x7f9a200b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a2859a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a201d8c50 0x7f9a201d90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a2859a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9a200af0d0 con 0x7f9a200b0f30 2024-03-16T10:11:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a2859a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9a200aef50 con 0x7f9a201d8c50 2024-03-16T10:11:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a2859a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9a200b4e50 con 0x7f9a0c005220 2024-03-16T10:11:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a25b35640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a200b0f30 0x7f9a200b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a25b35640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a200b0f30 0x7f9a200b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55750/0 (socket says 172.21.15.17:55750) 2024-03-16T10:11:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a25b35640 1 -- 172.21.15.17:0/828950980 learned_addr learned my addr 172.21.15.17:0/828950980 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a26336640 1 --2- 172.21.15.17:0/828950980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a0c005220 0x7f9a2009cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a26b37640 1 --2- 172.21.15.17:0/828950980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a201d8c50 0x7f9a201d90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a25b35640 1 -- 172.21.15.17:0/828950980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a0c005220 msgr2=0x7f9a2009cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a25b35640 1 --2- 172.21.15.17:0/828950980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a0c005220 0x7f9a2009cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a25b35640 1 -- 172.21.15.17:0/828950980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a201d8c50 msgr2=0x7f9a201d90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a25b35640 1 --2- 172.21.15.17:0/828950980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a201d8c50 0x7f9a201d90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:16.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a25b35640 1 -- 172.21.15.17:0/828950980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a200b18d0 con 0x7f9a200b0f30 2024-03-16T10:11:16.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a25b35640 1 --2- 172.21.15.17:0/828950980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a200b0f30 0x7f9a200b1390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000d7a0 tx=0x7f9a1000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:16.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a16ffd640 1 -- 172.21.15.17:0/828950980 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10005270 con 0x7f9a200b0f30 2024-03-16T10:11:16.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a16ffd640 1 -- 172.21.15.17:0/828950980 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a10017070 con 0x7f9a200b0f30 2024-03-16T10:11:16.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a2859a640 1 -- 172.21.15.17:0/828950980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a201d9610 con 0x7f9a200b0f30 2024-03-16T10:11:16.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a16ffd640 1 -- 172.21.15.17:0/828950980 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1000b650 con 0x7f9a200b0f30 2024-03-16T10:11:16.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a2859a640 1 -- 172.21.15.17:0/828950980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a201dbde0 con 0x7f9a200b0f30 2024-03-16T10:11:16.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a2859a640 1 -- 172.21.15.17:0/828950980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a2009ddf0 con 0x7f9a200b0f30 2024-03-16T10:11:16.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a16ffd640 1 -- 172.21.15.17:0/828950980 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f9a1000b7f0 con 0x7f9a200b0f30 2024-03-16T10:11:16.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.880+0000 7f9a16ffd640 1 --2- 172.21.15.17:0/828950980 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a04067660 0x7f9a04069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:16.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.884+0000 7f9a16ffd640 1 -- 172.21.15.17:0/828950980 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9a10085640 con 0x7f9a200b0f30 2024-03-16T10:11:16.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.884+0000 7f9a26336640 1 --2- 172.21.15.17:0/828950980 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a04067660 0x7f9a04069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:16.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.884+0000 7f9a26336640 1 --2- 172.21.15.17:0/828950980 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a04067660 0x7f9a04069b20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009910 tx=0x7f9a18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:16.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:16.888+0000 7f9a16ffd640 1 -- 172.21.15.17:0/828950980 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9a2009ddf0 con 0x7f9a200b0f30 2024-03-16T10:11:17.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.044+0000 7f9a2859a640 1 -- 172.21.15.17:0/828950980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9a2009c260 con 0x7f9a200b0f30 2024-03-16T10:11:17.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.044+0000 7f9a16ffd640 1 -- 172.21.15.17:0/828950980 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f9a2009c260 con 0x7f9a200b0f30 2024-03-16T10:11:17.050 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:11:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.048+0000 7f9a2859a640 1 -- 172.21.15.17:0/828950980 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a04067660 msgr2=0x7f9a04069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.048+0000 7f9a2859a640 1 --2- 172.21.15.17:0/828950980 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a04067660 0x7f9a04069b20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009910 tx=0x7f9a18008040 comp rx=0 tx=0).stop 2024-03-16T10:11:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.048+0000 7f9a2859a640 1 -- 172.21.15.17:0/828950980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a200b0f30 msgr2=0x7f9a200b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.048+0000 7f9a2859a640 1 --2- 172.21.15.17:0/828950980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a200b0f30 0x7f9a200b1390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000d7a0 tx=0x7f9a1000dc60 comp rx=0 tx=0).stop 2024-03-16T10:11:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.048+0000 7f9a2859a640 1 -- 172.21.15.17:0/828950980 shutdown_connections 2024-03-16T10:11:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.048+0000 7f9a2859a640 1 --2- 172.21.15.17:0/828950980 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a04067660 0x7f9a04069b20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.048+0000 7f9a2859a640 1 --2- 172.21.15.17:0/828950980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a201d8c50 0x7f9a201d90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.048+0000 7f9a2859a640 1 --2- 172.21.15.17:0/828950980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a200b0f30 0x7f9a200b1390 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.048+0000 7f9a2859a640 1 --2- 172.21.15.17:0/828950980 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a0c005220 0x7f9a2009cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.048+0000 7f9a2859a640 1 -- 172.21.15.17:0/828950980 >> 172.21.15.17:0/828950980 conn(0x7f9a2006bd70 msgr2=0x7f9a20144b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.048+0000 7f9a2859a640 1 -- 172.21.15.17:0/828950980 shutdown_connections 2024-03-16T10:11:17.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.048+0000 7f9a2859a640 1 -- 172.21.15.17:0/828950980 wait complete. 2024-03-16T10:11:17.069 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:11:17.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4abd3da640 1 Processor -- start 2024-03-16T10:11:17.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4abd3da640 1 -- start start 2024-03-16T10:11:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4abd3da640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ab8067220 0x7f4ab814ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4abd3da640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ab814b4c0 con 0x7f4ab814b040 2024-03-16T10:11:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4abd3da640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ab8067600 con 0x7f4ab8147070 2024-03-16T10:11:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4abd3da640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4ab80b40f0 con 0x7f4ab8067220 2024-03-16T10:11:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab67fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4ab814b040 0x7f4ab8146960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49012/0 (socket says 172.21.15.17:49012) 2024-03-16T10:11:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab67fc640 1 -- 172.21.15.17:0/297195419 learned_addr learned my addr 172.21.15.17:0/297195419 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 -- 172.21.15.17:0/297195419 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 437586358 0 0) 0x7f4aa8001120 con 0x7f4ab814b040 2024-03-16T10:11:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 -- 172.21.15.17:0/297195419 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4aa0003530 con 0x7f4ab814b040 2024-03-16T10:11:17.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab6ffd640 1 --2- 172.21.15.17:0/297195419 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ab8067220 0x7f4ab814ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 -- 172.21.15.17:0/297195419 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1176012000 0 0) 0x7f4aa0003530 con 0x7f4ab814b040 2024-03-16T10:11:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 -- 172.21.15.17:0/297195419 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ab8067220 msgr2=0x7f4ab814ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 --2- 172.21.15.17:0/297195419 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ab8067220 0x7f4ab814ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 -- 172.21.15.17:0/297195419 >> v1:172.21.15.174:6789/0 conn(0x7f4ab8147070 legacy=0x7f4ab8151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 -- 172.21.15.17:0/297195419 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab80b43d0 con 0x7f4ab814b040 2024-03-16T10:11:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 -- 172.21.15.17:0/297195419 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f4aa8001850 con 0x7f4ab814b040 2024-03-16T10:11:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 -- 172.21.15.17:0/297195419 >> v1:172.21.15.17:6789/0 conn(0x7f4ab814b040 legacy=0x7f4ab8146960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 --2- 172.21.15.17:0/297195419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aa00039f0 0x7f4aa0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 --2- 172.21.15.17:0/297195419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4aa0004810 0x7f4aa0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 --2- 172.21.15.17:0/297195419 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ab8067220 0x7f4aa0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 -- 172.21.15.17:0/297195419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab80b40f0 con 0x7f4aa00039f0 2024-03-16T10:11:17.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 -- 172.21.15.17:0/297195419 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa000bb00 con 0x7f4aa0004810 2024-03-16T10:11:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab5ffb640 1 -- 172.21.15.17:0/297195419 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4aa000bd00 con 0x7f4ab8067220 2024-03-16T10:11:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab6ffd640 1 --2- 172.21.15.17:0/297195419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aa00039f0 0x7f4aa0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab67fc640 1 --2- 172.21.15.17:0/297195419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4aa0004810 0x7f4aa0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab77fe640 1 --2- 172.21.15.17:0/297195419 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ab8067220 0x7f4aa0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab6ffd640 1 -- 172.21.15.17:0/297195419 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ab8067220 msgr2=0x7f4aa0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab6ffd640 1 --2- 172.21.15.17:0/297195419 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ab8067220 0x7f4aa0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab6ffd640 1 -- 172.21.15.17:0/297195419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4aa0004810 msgr2=0x7f4aa0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab6ffd640 1 --2- 172.21.15.17:0/297195419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4aa0004810 0x7f4aa0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab6ffd640 1 -- 172.21.15.17:0/297195419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa0003530 con 0x7f4aa00039f0 2024-03-16T10:11:17.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.180+0000 7f4ab77fe640 1 --2- 172.21.15.17:0/297195419 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4ab8067220 0x7f4aa0005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:17.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4ab6ffd640 1 --2- 172.21.15.17:0/297195419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aa00039f0 0x7f4aa0003e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4aac009ac0 tx=0x7f4aac01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7ba254fdd0262b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:17.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4ab5ffb640 1 -- 172.21.15.17:0/297195419 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0b2070 con 0x7f4aa00039f0 2024-03-16T10:11:17.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4ab5ffb640 1 -- 172.21.15.17:0/297195419 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aac0a5530 con 0x7f4aa00039f0 2024-03-16T10:11:17.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4ab5ffb640 1 -- 172.21.15.17:0/297195419 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac009690 con 0x7f4aa00039f0 2024-03-16T10:11:17.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4abd3da640 1 -- 172.21.15.17:0/297195419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aa00039f0 msgr2=0x7f4aa0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4abd3da640 1 --2- 172.21.15.17:0/297195419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4aa00039f0 0x7f4aa0003e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4aac009ac0 tx=0x7f4aac01ca50 comp rx=0 tx=0).stop 2024-03-16T10:11:17.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4ab77fe640 1 -- 172.21.15.17:0/297195419 reap_dead start 2024-03-16T10:11:17.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4abd3da640 1 -- 172.21.15.17:0/297195419 shutdown_connections 2024-03-16T10:11:17.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4abd3da640 1 -- 172.21.15.17:0/297195419 >> 172.21.15.17:0/297195419 conn(0x7f4ab806bd70 msgr2=0x7f4ab80b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:17.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4abd3da640 1 -- 172.21.15.17:0/297195419 shutdown_connections 2024-03-16T10:11:17.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4abd3da640 1 -- 172.21.15.17:0/297195419 wait complete. 2024-03-16T10:11:17.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4abd3da640 1 Processor -- start 2024-03-16T10:11:17.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4abd3da640 1 -- start start 2024-03-16T10:11:17.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4abd3da640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4aa00039f0 0x7f4ab8142130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4abd3da640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ab81e0470 0x7f4ab809d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4abd3da640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ab8071fa0 0x7f4ab8072420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4abd3da640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab80b3ca0 con 0x7f4ab8071fa0 2024-03-16T10:11:17.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4abd3da640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab80b5ad0 con 0x7f4ab81e0470 2024-03-16T10:11:17.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4abd3da640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4ab80b3e20 con 0x7f4aa00039f0 2024-03-16T10:11:17.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4ab77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ab8071fa0 0x7f4ab8072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4ab77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ab8071fa0 0x7f4ab8072420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55754/0 (socket says 172.21.15.17:55754) 2024-03-16T10:11:17.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4ab77fe640 1 -- 172.21.15.17:0/1610416012 learned_addr learned my addr 172.21.15.17:0/1610416012 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:17.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4ab6ffd640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4aa00039f0 0x7f4ab8142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.184+0000 7f4ab67fc640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ab81e0470 0x7f4ab809d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4ab77fe640 1 -- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4aa00039f0 msgr2=0x7f4ab8142130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4ab77fe640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4aa00039f0 0x7f4ab8142130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4ab77fe640 1 -- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ab81e0470 msgr2=0x7f4ab809d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4ab77fe640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ab81e0470 0x7f4ab809d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4ab77fe640 1 -- 172.21.15.17:0/1610416012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab8072960 con 0x7f4ab8071fa0 2024-03-16T10:11:17.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4ab67fc640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ab81e0470 0x7f4ab809d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:17.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4ab77fe640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ab8071fa0 0x7f4ab8072420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0004790 tx=0x7f4ab000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:17.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4a9f7fe640 1 -- 172.21.15.17:0/1610416012 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0006d60 con 0x7f4ab8071fa0 2024-03-16T10:11:17.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4a9f7fe640 1 -- 172.21.15.17:0/1610416012 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ab0003980 con 0x7f4ab8071fa0 2024-03-16T10:11:17.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4abd3da640 1 -- 172.21.15.17:0/1610416012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab8072d60 con 0x7f4ab8071fa0 2024-03-16T10:11:17.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4a9f7fe640 1 -- 172.21.15.17:0/1610416012 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0003d70 con 0x7f4ab8071fa0 2024-03-16T10:11:17.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4abd3da640 1 -- 172.21.15.17:0/1610416012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab809cc70 con 0x7f4ab8071fa0 2024-03-16T10:11:17.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4abd3da640 1 -- 172.21.15.17:0/1610416012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ab81e13d0 con 0x7f4ab8071fa0 2024-03-16T10:11:17.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4a9f7fe640 1 -- 172.21.15.17:0/1610416012 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f4ab0005070 con 0x7f4ab8071fa0 2024-03-16T10:11:17.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4a9f7fe640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4aa4067690 0x7f4aa4069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4a9f7fe640 1 -- 172.21.15.17:0/1610416012 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4ab0085560 con 0x7f4ab8071fa0 2024-03-16T10:11:17.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4ab6ffd640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4aa4067690 0x7f4aa4069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.188+0000 7f4ab6ffd640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4aa4067690 0x7f4aa4069b50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0af000 tx=0x7f4aac0046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:17.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.192+0000 7f4a9f7fe640 1 -- 172.21.15.17:0/1610416012 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4ab0056270 con 0x7f4ab8071fa0 2024-03-16T10:11:17.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.356+0000 7f4abd3da640 1 -- 172.21.15.17:0/1610416012 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4ab8149d90 con 0x7f4ab8071fa0 2024-03-16T10:11:17.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.356+0000 7f4a9f7fe640 1 -- 172.21.15.17:0/1610416012 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f4ab0055b70 con 0x7f4ab8071fa0 2024-03-16T10:11:17.362 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-16T10:11:17.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.360+0000 7f4abd3da640 1 -- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4aa4067690 msgr2=0x7f4aa4069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.360+0000 7f4abd3da640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4aa4067690 0x7f4aa4069b50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4aac0af000 tx=0x7f4aac0046d0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.360+0000 7f4abd3da640 1 -- 172.21.15.17:0/1610416012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ab8071fa0 msgr2=0x7f4ab8072420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.360+0000 7f4abd3da640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ab8071fa0 0x7f4ab8072420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0004790 tx=0x7f4ab000dc00 comp rx=0 tx=0).stop 2024-03-16T10:11:17.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.360+0000 7f4abd3da640 1 -- 172.21.15.17:0/1610416012 shutdown_connections 2024-03-16T10:11:17.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.360+0000 7f4abd3da640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4aa4067690 0x7f4aa4069b50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.360+0000 7f4abd3da640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ab8071fa0 0x7f4ab8072420 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.360+0000 7f4abd3da640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ab81e0470 0x7f4ab809d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.360+0000 7f4abd3da640 1 --2- 172.21.15.17:0/1610416012 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4aa00039f0 0x7f4ab8142130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.360+0000 7f4abd3da640 1 -- 172.21.15.17:0/1610416012 >> 172.21.15.17:0/1610416012 conn(0x7f4ab806bd70 msgr2=0x7f4ab8143cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:17.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.360+0000 7f4abd3da640 1 -- 172.21.15.17:0/1610416012 shutdown_connections 2024-03-16T10:11:17.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.360+0000 7f4abd3da640 1 -- 172.21.15.17:0/1610416012 wait complete. 2024-03-16T10:11:17.379 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:11:17.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebfabb640 1 Processor -- start 2024-03-16T10:11:17.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebfabb640 1 -- start start 2024-03-16T10:11:17.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebfabb640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7efeb80b5740 0x7efeb80aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebfabb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efeb806ede0 0x7efeb8071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebfabb640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb80b5b20 con 0x7efeb8072410 2024-03-16T10:11:17.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebfabb640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7efeb806e950 con 0x7efeb80b5740 2024-03-16T10:11:17.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebfabb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7efeb80af720 con 0x7efeb806ede0 2024-03-16T10:11:17.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebd056640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efeb806ede0 0x7efeb8071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebd857640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7efeb80b5740 0x7efeb80aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebd056640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efeb806ede0 0x7efeb8071ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.17:34498/0 (socket says 172.21.15.17:34498) 2024-03-16T10:11:17.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebd857640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7efeb80b5740 0x7efeb80aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.17:34480/0 (socket says 172.21.15.17:34480) 2024-03-16T10:11:17.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebd056640 1 -- 172.21.15.17:0/1996526526 learned_addr learned my addr 172.21.15.17:0/1996526526 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:17.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebd056640 1 -- 172.21.15.17:0/1996526526 >> v2:172.21.15.174:3300/0 conn(0x7efeb80b5740 msgr2=0x7efeb80aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:17.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebd056640 1 --2- 172.21.15.17:0/1996526526 >> v2:172.21.15.174:3300/0 conn(0x7efeb80b5740 0x7efeb80aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebd056640 1 -- 172.21.15.17:0/1996526526 >> v1:172.21.15.174:6789/0 conn(0x7efeb8072410 legacy=0x7efeb81440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebd056640 1 -- 172.21.15.17:0/1996526526 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb80afa00 con 0x7efeb806ede0 2024-03-16T10:11:17.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.492+0000 7efebd857640 1 --2- 172.21.15.17:0/1996526526 >> v2:172.21.15.174:3300/0 conn(0x7efeb80b5740 0x7efeb80aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:11:17.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.496+0000 7efebd056640 1 --2- 172.21.15.17:0/1996526526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efeb806ede0 0x7efeb8071ed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efea8009870 tx=0x7efea801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e98c54f45228195 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:17.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.496+0000 7efebc855640 1 -- 172.21.15.17:0/1996526526 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea80a7070 con 0x7efeb806ede0 2024-03-16T10:11:17.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.496+0000 7efebc855640 1 -- 172.21.15.17:0/1996526526 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea80b0070 con 0x7efeb806ede0 2024-03-16T10:11:17.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.496+0000 7efebc855640 1 -- 172.21.15.17:0/1996526526 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea80ab890 con 0x7efeb806ede0 2024-03-16T10:11:17.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.496+0000 7efebfabb640 1 -- 172.21.15.17:0/1996526526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efeb806ede0 msgr2=0x7efeb8071ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.496+0000 7efebfabb640 1 --2- 172.21.15.17:0/1996526526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efeb806ede0 0x7efeb8071ed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efea8009870 tx=0x7efea801c560 comp rx=0 tx=0).stop 2024-03-16T10:11:17.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.496+0000 7efebfabb640 1 -- 172.21.15.17:0/1996526526 shutdown_connections 2024-03-16T10:11:17.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.496+0000 7efebfabb640 1 --2- 172.21.15.17:0/1996526526 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efeb806ede0 0x7efeb8071ed0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.496+0000 7efebfabb640 1 --2- 172.21.15.17:0/1996526526 >> v2:172.21.15.174:3300/0 conn(0x7efeb80b5740 0x7efeb80aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.496+0000 7efebfabb640 1 -- 172.21.15.17:0/1996526526 >> 172.21.15.17:0/1996526526 conn(0x7efeb80b65d0 msgr2=0x7efeb80b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:17.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.496+0000 7efebfabb640 1 -- 172.21.15.17:0/1996526526 shutdown_connections 2024-03-16T10:11:17.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.496+0000 7efebfabb640 1 -- 172.21.15.17:0/1996526526 wait complete. 2024-03-16T10:11:17.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebfabb640 1 Processor -- start 2024-03-16T10:11:17.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebfabb640 1 -- start start 2024-03-16T10:11:17.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebfabb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efeb806ede0 0x7efeb81347e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebfabb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efeb8072410 0x7efeb8134d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebfabb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb80b5740 0x7efeb8141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebfabb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81477a0 con 0x7efeb80b5740 2024-03-16T10:11:17.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebfabb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efeb8147620 con 0x7efeb8072410 2024-03-16T10:11:17.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebfabb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7efeb806b290 con 0x7efeb806ede0 2024-03-16T10:11:17.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebe058640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb80b5740 0x7efeb8141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebe058640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb80b5740 0x7efeb8141f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55766/0 (socket says 172.21.15.17:55766) 2024-03-16T10:11:17.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebe058640 1 -- 172.21.15.17:0/3179418303 learned_addr learned my addr 172.21.15.17:0/3179418303 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:17.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebd056640 1 --2- 172.21.15.17:0/3179418303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efeb8072410 0x7efeb8134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebd857640 1 --2- 172.21.15.17:0/3179418303 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efeb806ede0 0x7efeb81347e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebe058640 1 -- 172.21.15.17:0/3179418303 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efeb806ede0 msgr2=0x7efeb81347e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebe058640 1 --2- 172.21.15.17:0/3179418303 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efeb806ede0 0x7efeb81347e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebe058640 1 -- 172.21.15.17:0/3179418303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efeb8072410 msgr2=0x7efeb8134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebe058640 1 --2- 172.21.15.17:0/3179418303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efeb8072410 0x7efeb8134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebe058640 1 -- 172.21.15.17:0/3179418303 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb81e0200 con 0x7efeb80b5740 2024-03-16T10:11:17.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebe058640 1 --2- 172.21.15.17:0/3179418303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb80b5740 0x7efeb8141f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efeb000c770 tx=0x7efeb000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:17.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efeae7fc640 1 -- 172.21.15.17:0/3179418303 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb0003b80 con 0x7efeb80b5740 2024-03-16T10:11:17.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efeae7fc640 1 -- 172.21.15.17:0/3179418303 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeb0003d20 con 0x7efeb80b5740 2024-03-16T10:11:17.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebfabb640 1 -- 172.21.15.17:0/3179418303 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb81e04e0 con 0x7efeb80b5740 2024-03-16T10:11:17.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efeae7fc640 1 -- 172.21.15.17:0/3179418303 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb000aa90 con 0x7efeb80b5740 2024-03-16T10:11:17.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebfabb640 1 -- 172.21.15.17:0/3179418303 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb8139370 con 0x7efeb80b5740 2024-03-16T10:11:17.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efebfabb640 1 -- 172.21.15.17:0/3179418303 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb806a130 con 0x7efeb80b5740 2024-03-16T10:11:17.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.500+0000 7efeae7fc640 1 -- 172.21.15.17:0/3179418303 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7efeb000ac30 con 0x7efeb80b5740 2024-03-16T10:11:17.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.504+0000 7efeae7fc640 1 --2- 172.21.15.17:0/3179418303 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efe9c067660 0x7efe9c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.504+0000 7efeae7fc640 1 -- 172.21.15.17:0/3179418303 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efeb00855e0 con 0x7efeb80b5740 2024-03-16T10:11:17.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.504+0000 7efebd857640 1 --2- 172.21.15.17:0/3179418303 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efe9c067660 0x7efe9c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.504+0000 7efebd857640 1 --2- 172.21.15.17:0/3179418303 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efe9c067660 0x7efe9c069b20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efeb4009580 tx=0x7efeb4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:17.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.508+0000 7efeae7fc640 1 -- 172.21.15.17:0/3179418303 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7efeb0055ca0 con 0x7efeb80b5740 2024-03-16T10:11:17.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.664+0000 7efebfabb640 1 -- 172.21.15.17:0/3179418303 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efeb8144370 con 0x7efeb80b5740 2024-03-16T10:11:17.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.664+0000 7efeae7fc640 1 -- 172.21.15.17:0/3179418303 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7efeb0059d40 con 0x7efeb80b5740 2024-03-16T10:11:17.669 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-16T10:11:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.664+0000 7efebfabb640 1 -- 172.21.15.17:0/3179418303 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efe9c067660 msgr2=0x7efe9c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.664+0000 7efebfabb640 1 --2- 172.21.15.17:0/3179418303 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efe9c067660 0x7efe9c069b20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efeb4009580 tx=0x7efeb4009190 comp rx=0 tx=0).stop 2024-03-16T10:11:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.664+0000 7efebfabb640 1 -- 172.21.15.17:0/3179418303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb80b5740 msgr2=0x7efeb8141f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.664+0000 7efebfabb640 1 --2- 172.21.15.17:0/3179418303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb80b5740 0x7efeb8141f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7efeb000c770 tx=0x7efeb000cc30 comp rx=0 tx=0).stop 2024-03-16T10:11:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.664+0000 7efebfabb640 1 -- 172.21.15.17:0/3179418303 shutdown_connections 2024-03-16T10:11:17.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.664+0000 7efebfabb640 1 --2- 172.21.15.17:0/3179418303 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efe9c067660 0x7efe9c069b20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.664+0000 7efebfabb640 1 --2- 172.21.15.17:0/3179418303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb80b5740 0x7efeb8141f20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.664+0000 7efebfabb640 1 --2- 172.21.15.17:0/3179418303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efeb8072410 0x7efeb8134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.664+0000 7efebfabb640 1 --2- 172.21.15.17:0/3179418303 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efeb806ede0 0x7efeb81347e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.664+0000 7efebfabb640 1 -- 172.21.15.17:0/3179418303 >> 172.21.15.17:0/3179418303 conn(0x7efeb80b65d0 msgr2=0x7efeb809ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.664+0000 7efebfabb640 1 -- 172.21.15.17:0/3179418303 shutdown_connections 2024-03-16T10:11:17.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.664+0000 7efebfabb640 1 -- 172.21.15.17:0/3179418303 wait complete. 2024-03-16T10:11:17.686 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:17.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.788+0000 7fc3b9e5c640 1 Processor -- start 2024-03-16T10:11:17.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.788+0000 7fc3b9e5c640 1 -- start start 2024-03-16T10:11:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b9e5c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc3b40b6700 0x7fc3b409f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b9e5c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3b406b9a0 0x7fc3b41520d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b9e5c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3b40b6ae0 con 0x7fc3b409fd30 2024-03-16T10:11:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b9e5c640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b40b4d60 con 0x7fc3b406b9a0 2024-03-16T10:11:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b9e5c640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fc3b40b5600 con 0x7fc3b40b6700 2024-03-16T10:11:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b3fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc3b406b9a0 0x7fc3b41520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b2ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc3b409fd30 0x7fc3b406b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:49016/0 (socket says 172.21.15.17:49016) 2024-03-16T10:11:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b2ffd640 1 -- 172.21.15.17:0/2762173382 learned_addr learned my addr 172.21.15.17:0/2762173382 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b37fe640 1 --2- 172.21.15.17:0/2762173382 >> v2:172.21.15.174:3300/0 conn(0x7fc3b40b6700 0x7fc3b409f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b3fff640 1 -- 172.21.15.17:0/2762173382 >> v2:172.21.15.174:3300/0 conn(0x7fc3b40b6700 msgr2=0x7fc3b409f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b3fff640 1 --2- 172.21.15.17:0/2762173382 >> v2:172.21.15.174:3300/0 conn(0x7fc3b40b6700 0x7fc3b409f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b3fff640 1 -- 172.21.15.17:0/2762173382 >> v1:172.21.15.17:6789/0 conn(0x7fc3b409fd30 legacy=0x7fc3b406b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b27fc640 1 -- 172.21.15.17:0/2762173382 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3986950573 0 0) 0x7fc3b40b6ae0 con 0x7fc3b409fd30 2024-03-16T10:11:17.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b3fff640 1 -- 172.21.15.17:0/2762173382 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b40aefe0 con 0x7fc3b406b9a0 2024-03-16T10:11:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b3fff640 1 --2- 172.21.15.17:0/2762173382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b406b9a0 0x7fc3b41520d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40b4160 tx=0x7fc3ac01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d1f938eb3d91e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b27fc640 1 -- 172.21.15.17:0/2762173382 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac0b2070 con 0x7fc3b406b9a0 2024-03-16T10:11:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b27fc640 1 -- 172.21.15.17:0/2762173382 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3ac0a5930 con 0x7fc3b406b9a0 2024-03-16T10:11:17.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b27fc640 1 -- 172.21.15.17:0/2762173382 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac0a5d00 con 0x7fc3b406b9a0 2024-03-16T10:11:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2762173382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b406b9a0 msgr2=0x7fc3b41520d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b9e5c640 1 --2- 172.21.15.17:0/2762173382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b406b9a0 0x7fc3b41520d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc3b40b4160 tx=0x7fc3ac01c870 comp rx=0 tx=0).stop 2024-03-16T10:11:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2762173382 shutdown_connections 2024-03-16T10:11:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b9e5c640 1 --2- 172.21.15.17:0/2762173382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b406b9a0 0x7fc3b41520d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b9e5c640 1 --2- 172.21.15.17:0/2762173382 >> v2:172.21.15.174:3300/0 conn(0x7fc3b40b6700 0x7fc3b409f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2762173382 >> 172.21.15.17:0/2762173382 conn(0x7fc3b406e620 msgr2=0x7fc3b406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:17.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2762173382 shutdown_connections 2024-03-16T10:11:17.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.792+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2762173382 wait complete. 2024-03-16T10:11:17.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b9e5c640 1 Processor -- start 2024-03-16T10:11:17.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b9e5c640 1 -- start start 2024-03-16T10:11:17.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b9e5c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b406b9a0 0x7fc3b40b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b9e5c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3b409fd30 0x7fc3b40b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b9e5c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3b40b6700 0x7fc3b41e0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b9e5c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b40b5d90 con 0x7fc3b406b9a0 2024-03-16T10:11:17.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b9e5c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b40af470 con 0x7fc3b40b6700 2024-03-16T10:11:17.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b9e5c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc3b4141180 con 0x7fc3b409fd30 2024-03-16T10:11:17.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b406b9a0 0x7fc3b40b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b406b9a0 0x7fc3b40b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55786/0 (socket says 172.21.15.17:55786) 2024-03-16T10:11:17.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b37fe640 1 -- 172.21.15.17:0/2046543444 learned_addr learned my addr 172.21.15.17:0/2046543444 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:17.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b3fff640 1 --2- 172.21.15.17:0/2046543444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3b40b6700 0x7fc3b41e0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b2ffd640 1 --2- 172.21.15.17:0/2046543444 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3b409fd30 0x7fc3b40b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b37fe640 1 -- 172.21.15.17:0/2046543444 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3b409fd30 msgr2=0x7fc3b40b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b37fe640 1 --2- 172.21.15.17:0/2046543444 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3b409fd30 0x7fc3b40b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b37fe640 1 -- 172.21.15.17:0/2046543444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3b40b6700 msgr2=0x7fc3b41e0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b37fe640 1 --2- 172.21.15.17:0/2046543444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3b40b6700 0x7fc3b41e0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b37fe640 1 -- 172.21.15.17:0/2046543444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b40b3a90 con 0x7fc3b406b9a0 2024-03-16T10:11:17.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b37fe640 1 --2- 172.21.15.17:0/2046543444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b406b9a0 0x7fc3b40b1fc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800d5f0 tx=0x7fc3a800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:17.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc39bfff640 1 -- 172.21.15.17:0/2046543444 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80125f0 con 0x7fc3b406b9a0 2024-03-16T10:11:17.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc39bfff640 1 -- 172.21.15.17:0/2046543444 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a8019070 con 0x7fc3b406b9a0 2024-03-16T10:11:17.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc39bfff640 1 -- 172.21.15.17:0/2046543444 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80129d0 con 0x7fc3b406b9a0 2024-03-16T10:11:17.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2046543444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b40b0f80 con 0x7fc3b406b9a0 2024-03-16T10:11:17.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2046543444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b40b37a0 con 0x7fc3b406b9a0 2024-03-16T10:11:17.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2046543444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b406a130 con 0x7fc3b406b9a0 2024-03-16T10:11:17.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc39bfff640 1 -- 172.21.15.17:0/2046543444 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fc3a8012b70 con 0x7fc3b406b9a0 2024-03-16T10:11:17.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.796+0000 7fc39bfff640 1 --2- 172.21.15.17:0/2046543444 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc3a0067660 0x7fc3a0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:17.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.800+0000 7fc39bfff640 1 -- 172.21.15.17:0/2046543444 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc3a8085c60 con 0x7fc3b406b9a0 2024-03-16T10:11:17.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.800+0000 7fc3b2ffd640 1 --2- 172.21.15.17:0/2046543444 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc3a0067660 0x7fc3a0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:17.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.800+0000 7fc3b2ffd640 1 --2- 172.21.15.17:0/2046543444 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc3a0067660 0x7fc3a0069b20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4148420 tx=0x7fc3a4007bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:17.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.804+0000 7fc39bfff640 1 -- 172.21.15.17:0/2046543444 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc3a8056970 con 0x7fc3b406b9a0 2024-03-16T10:11:17.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.956+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2046543444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3b4068a40 con 0x7fc3b406b9a0 2024-03-16T10:11:17.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.960+0000 7fc39bfff640 1 -- 172.21.15.17:0/2046543444 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7fc3a8056270 con 0x7fc3b406b9a0 2024-03-16T10:11:17.964 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:17.965 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:17.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.960+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2046543444 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc3a0067660 msgr2=0x7fc3a0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.960+0000 7fc3b9e5c640 1 --2- 172.21.15.17:0/2046543444 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc3a0067660 0x7fc3a0069b20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4148420 tx=0x7fc3a4007bc0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.960+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2046543444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b406b9a0 msgr2=0x7fc3b40b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:17.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.960+0000 7fc3b9e5c640 1 --2- 172.21.15.17:0/2046543444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b406b9a0 0x7fc3b40b1fc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800d5f0 tx=0x7fc3a800dab0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.960+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2046543444 shutdown_connections 2024-03-16T10:11:17.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.960+0000 7fc3b9e5c640 1 --2- 172.21.15.17:0/2046543444 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc3a0067660 0x7fc3a0069b20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.960+0000 7fc3b9e5c640 1 --2- 172.21.15.17:0/2046543444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3b40b6700 0x7fc3b41e0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.960+0000 7fc3b9e5c640 1 --2- 172.21.15.17:0/2046543444 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc3b409fd30 0x7fc3b40b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.960+0000 7fc3b9e5c640 1 --2- 172.21.15.17:0/2046543444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc3b406b9a0 0x7fc3b40b1fc0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:17.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.960+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2046543444 >> 172.21.15.17:0/2046543444 conn(0x7fc3b406e620 msgr2=0x7fc3b406ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:17.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.960+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2046543444 shutdown_connections 2024-03-16T10:11:17.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:17.960+0000 7fc3b9e5c640 1 -- 172.21.15.17:0/2046543444 wait complete. 2024-03-16T10:11:17.982 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:10:55.723734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:10:56.459894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:11:17.982 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:11:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a73c2640 1 Processor -- start 2024-03-16T10:11:18.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a73c2640 1 -- start start 2024-03-16T10:11:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a73c2640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f65a006e3c0 0x7f65a00af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a73c2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f65a00af8d0 0x7f65a0147d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a73c2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a006e7a0 con 0x7f65a01482a0 2024-03-16T10:11:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a73c2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f65a009f260 con 0x7f65a00af8d0 2024-03-16T10:11:18.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a73c2640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f65a00aec20 con 0x7f65a006e3c0 2024-03-16T10:11:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a495d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f65a00af8d0 0x7f65a0147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a495d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f65a00af8d0 0x7f65a0147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55794/0 (socket says 172.21.15.17:55794) 2024-03-16T10:11:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a495d640 1 -- 172.21.15.17:0/2035886143 learned_addr learned my addr 172.21.15.17:0/2035886143 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a515e640 1 --2- 172.21.15.17:0/2035886143 >> v2:172.21.15.174:3300/0 conn(0x7f65a006e3c0 0x7f65a00af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a495d640 1 -- 172.21.15.17:0/2035886143 >> v2:172.21.15.174:3300/0 conn(0x7f65a006e3c0 msgr2=0x7f65a00af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a495d640 1 --2- 172.21.15.17:0/2035886143 >> v2:172.21.15.174:3300/0 conn(0x7f65a006e3c0 0x7f65a00af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a495d640 1 -- 172.21.15.17:0/2035886143 >> v1:172.21.15.17:6789/0 conn(0x7f65a01482a0 legacy=0x7f65a00b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a495d640 1 -- 172.21.15.17:0/2035886143 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a006dd10 con 0x7f65a00af8d0 2024-03-16T10:11:18.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.108+0000 7f65a515e640 1 --2- 172.21.15.17:0/2035886143 >> v2:172.21.15.174:3300/0 conn(0x7f65a006e3c0 0x7f65a00af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:18.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.112+0000 7f65a495d640 1 --2- 172.21.15.17:0/2035886143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65a00af8d0 0x7f65a0147d60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f65a00b99e0 tx=0x7f659401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be06b327dba011ac server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.112+0000 7f659ffff640 1 -- 172.21.15.17:0/2035886143 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65940b0070 con 0x7f65a00af8d0 2024-03-16T10:11:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.112+0000 7f659ffff640 1 -- 172.21.15.17:0/2035886143 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65940ab600 con 0x7f65a00af8d0 2024-03-16T10:11:18.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.112+0000 7f659ffff640 1 -- 172.21.15.17:0/2035886143 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65940ab9d0 con 0x7f65a00af8d0 2024-03-16T10:11:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.112+0000 7f65a73c2640 1 -- 172.21.15.17:0/2035886143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65a00af8d0 msgr2=0x7f65a0147d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.112+0000 7f65a73c2640 1 --2- 172.21.15.17:0/2035886143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65a00af8d0 0x7f65a0147d60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f65a00b99e0 tx=0x7f659401c520 comp rx=0 tx=0).stop 2024-03-16T10:11:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.112+0000 7f65a73c2640 1 -- 172.21.15.17:0/2035886143 shutdown_connections 2024-03-16T10:11:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.112+0000 7f65a73c2640 1 --2- 172.21.15.17:0/2035886143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65a00af8d0 0x7f65a0147d60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.112+0000 7f65a73c2640 1 --2- 172.21.15.17:0/2035886143 >> v2:172.21.15.174:3300/0 conn(0x7f65a006e3c0 0x7f65a00af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.112+0000 7f65a73c2640 1 -- 172.21.15.17:0/2035886143 >> 172.21.15.17:0/2035886143 conn(0x7f65a00b65d0 msgr2=0x7f65a00b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.112+0000 7f65a73c2640 1 -- 172.21.15.17:0/2035886143 shutdown_connections 2024-03-16T10:11:18.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.112+0000 7f65a73c2640 1 -- 172.21.15.17:0/2035886143 wait complete. 2024-03-16T10:11:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.112+0000 7f65a73c2640 1 Processor -- start 2024-03-16T10:11:18.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.112+0000 7f65a73c2640 1 -- start start 2024-03-16T10:11:18.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a73c2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a006e3c0 0x7f65a01e38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a73c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65a00af8d0 0x7f65a0146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a73c2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a01482a0 0x7f65a0145070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a73c2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f65a006f070 con 0x7f65a00af8d0 2024-03-16T10:11:18.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a73c2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65a01486b0 con 0x7f65a006e3c0 2024-03-16T10:11:18.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a73c2640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f65a006d4d0 con 0x7f65a01482a0 2024-03-16T10:11:18.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a495d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65a00af8d0 0x7f65a0146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a495d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65a00af8d0 0x7f65a0146340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55798/0 (socket says 172.21.15.17:55798) 2024-03-16T10:11:18.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a495d640 1 -- 172.21.15.17:0/3746674428 learned_addr learned my addr 172.21.15.17:0/3746674428 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:18.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a495d640 1 -- 172.21.15.17:0/3746674428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a01482a0 msgr2=0x7f65a0145070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a495d640 1 --2- 172.21.15.17:0/3746674428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a01482a0 0x7f65a0145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a495d640 1 -- 172.21.15.17:0/3746674428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a006e3c0 msgr2=0x7f65a01e38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a495d640 1 --2- 172.21.15.17:0/3746674428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a006e3c0 0x7f65a01e38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a495d640 1 -- 172.21.15.17:0/3746674428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a0148d60 con 0x7f65a00af8d0 2024-03-16T10:11:18.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a495d640 1 --2- 172.21.15.17:0/3746674428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65a00af8d0 0x7f65a0146340 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6594019130 tx=0x7f65940192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:18.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f659dffb640 1 -- 172.21.15.17:0/3746674428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65940b9070 con 0x7f65a00af8d0 2024-03-16T10:11:18.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f659dffb640 1 -- 172.21.15.17:0/3746674428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65940b4600 con 0x7f65a00af8d0 2024-03-16T10:11:18.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a73c2640 1 -- 172.21.15.17:0/3746674428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a0149320 con 0x7f65a00af8d0 2024-03-16T10:11:18.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f659dffb640 1 -- 172.21.15.17:0/3746674428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65940b0040 con 0x7f65a00af8d0 2024-03-16T10:11:18.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a73c2640 1 -- 172.21.15.17:0/3746674428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a014b7e0 con 0x7f65a00af8d0 2024-03-16T10:11:18.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a73c2640 1 -- 172.21.15.17:0/3746674428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a006a130 con 0x7f65a00af8d0 2024-03-16T10:11:18.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f659dffb640 1 -- 172.21.15.17:0/3746674428 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f65940bc070 con 0x7f65a00af8d0 2024-03-16T10:11:18.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f659dffb640 1 --2- 172.21.15.17:0/3746674428 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f658c067690 0x7f658c069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f659dffb640 1 -- 172.21.15.17:0/3746674428 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f659411ee80 con 0x7f65a00af8d0 2024-03-16T10:11:18.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.116+0000 7f65a515e640 1 --2- 172.21.15.17:0/3746674428 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f658c067690 0x7f658c069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.120+0000 7f65a515e640 1 --2- 172.21.15.17:0/3746674428 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f658c067690 0x7f658c069b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6590008ea0 tx=0x7f65900083d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:18.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.120+0000 7f659dffb640 1 -- 172.21.15.17:0/3746674428 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6594016110 con 0x7f65a00af8d0 2024-03-16T10:11:18.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.276+0000 7f65a73c2640 1 -- 172.21.15.17:0/3746674428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f65a0149040 con 0x7f65a00af8d0 2024-03-16T10:11:18.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.276+0000 7f659dffb640 1 -- 172.21.15.17:0/3746674428 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f65940efbf0 con 0x7f65a00af8d0 2024-03-16T10:11:18.282 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:11:18.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.280+0000 7f65a73c2640 1 -- 172.21.15.17:0/3746674428 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f658c067690 msgr2=0x7f658c069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.280+0000 7f65a73c2640 1 --2- 172.21.15.17:0/3746674428 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f658c067690 0x7f658c069b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6590008ea0 tx=0x7f65900083d0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.280+0000 7f65a73c2640 1 -- 172.21.15.17:0/3746674428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65a00af8d0 msgr2=0x7f65a0146340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.280+0000 7f65a73c2640 1 --2- 172.21.15.17:0/3746674428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65a00af8d0 0x7f65a0146340 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6594019130 tx=0x7f65940192c0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.280+0000 7f65a73c2640 1 -- 172.21.15.17:0/3746674428 shutdown_connections 2024-03-16T10:11:18.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.280+0000 7f65a73c2640 1 --2- 172.21.15.17:0/3746674428 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f658c067690 0x7f658c069b50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.280+0000 7f65a73c2640 1 --2- 172.21.15.17:0/3746674428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f65a01482a0 0x7f65a0145070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.280+0000 7f65a73c2640 1 --2- 172.21.15.17:0/3746674428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65a00af8d0 0x7f65a0146340 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.280+0000 7f65a73c2640 1 --2- 172.21.15.17:0/3746674428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65a006e3c0 0x7f65a01e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.280+0000 7f65a73c2640 1 -- 172.21.15.17:0/3746674428 >> 172.21.15.17:0/3746674428 conn(0x7f65a00b65d0 msgr2=0x7f65a00b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:18.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.280+0000 7f65a73c2640 1 -- 172.21.15.17:0/3746674428 shutdown_connections 2024-03-16T10:11:18.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.280+0000 7f65a73c2640 1 -- 172.21.15.17:0/3746674428 wait complete. 2024-03-16T10:11:18.299 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:11:18.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8abe99640 1 Processor -- start 2024-03-16T10:11:18.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8abe99640 1 -- start start 2024-03-16T10:11:18.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8abe99640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a40b7770 0x7fc8a4145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8abe99640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc8a4073070 0x7fc8a40b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8abe99640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8a40b6fd0 con 0x7fc8a4145740 2024-03-16T10:11:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8abe99640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fc8a40b64f0 con 0x7fc8a4073070 2024-03-16T10:11:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8abe99640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a40b67d0 con 0x7fc8a40b7770 2024-03-16T10:11:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8a9434640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc8a4073070 0x7fc8a40b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8a9c35640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a40b7770 0x7fc8a4145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8a9434640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc8a4073070 0x7fc8a40b42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.17:34544/0 (socket says 172.21.15.17:34544) 2024-03-16T10:11:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8a9434640 1 -- 172.21.15.17:0/1954282521 learned_addr learned my addr 172.21.15.17:0/1954282521 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8aa436640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7fc8a4145740 0x7fc8a4148770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.17:33700/0 (socket says 172.21.15.17:33700) 2024-03-16T10:11:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8a9434640 1 -- 172.21.15.17:0/1954282521 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a40b7770 msgr2=0x7fc8a4145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8a9434640 1 --2- 172.21.15.17:0/1954282521 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a40b7770 0x7fc8a4145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8a9434640 1 -- 172.21.15.17:0/1954282521 >> v1:172.21.15.174:6789/0 conn(0x7fc8a4145740 legacy=0x7fc8a4148770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8a9434640 1 -- 172.21.15.17:0/1954282521 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a40b6ab0 con 0x7fc8a4073070 2024-03-16T10:11:18.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.420+0000 7fc8a9c35640 1 --2- 172.21.15.17:0/1954282521 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a40b7770 0x7fc8a4145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:11:18.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8a9434640 1 --2- 172.21.15.17:0/1954282521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a4073070 0x7fc8a40b42c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc894009870 tx=0x7fc89401c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=741c06c4679241a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8a8c33640 1 -- 172.21.15.17:0/1954282521 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8940b0070 con 0x7fc8a4073070 2024-03-16T10:11:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8a8c33640 1 -- 172.21.15.17:0/1954282521 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc894004230 con 0x7fc8a4073070 2024-03-16T10:11:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8a8c33640 1 -- 172.21.15.17:0/1954282521 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894003620 con 0x7fc8a4073070 2024-03-16T10:11:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 -- 172.21.15.17:0/1954282521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a4073070 msgr2=0x7fc8a40b42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 --2- 172.21.15.17:0/1954282521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a4073070 0x7fc8a40b42c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc894009870 tx=0x7fc89401c440 comp rx=0 tx=0).stop 2024-03-16T10:11:18.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 -- 172.21.15.17:0/1954282521 shutdown_connections 2024-03-16T10:11:18.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 --2- 172.21.15.17:0/1954282521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a4073070 0x7fc8a40b42c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 --2- 172.21.15.17:0/1954282521 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a40b7770 0x7fc8a4145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 -- 172.21.15.17:0/1954282521 >> 172.21.15.17:0/1954282521 conn(0x7fc8a406b290 msgr2=0x7fc8a406b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:18.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 -- 172.21.15.17:0/1954282521 shutdown_connections 2024-03-16T10:11:18.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 -- 172.21.15.17:0/1954282521 wait complete. 2024-03-16T10:11:18.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 Processor -- start 2024-03-16T10:11:18.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 -- start start 2024-03-16T10:11:18.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a4073070 0x7fc8a414e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a40b7770 0x7fc8a414f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8a4145740 0x7fc8a414eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4152c40 con 0x7fc8a4145740 2024-03-16T10:11:18.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4152ac0 con 0x7fc8a4073070 2024-03-16T10:11:18.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8abe99640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc8a4152dc0 con 0x7fc8a40b7770 2024-03-16T10:11:18.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8aa436640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8a4145740 0x7fc8a414eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8aa436640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8a4145740 0x7fc8a414eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55812/0 (socket says 172.21.15.17:55812) 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8aa436640 1 -- 172.21.15.17:0/2972704860 learned_addr learned my addr 172.21.15.17:0/2972704860 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8a9434640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a40b7770 0x7fc8a414f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8a9c35640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a4073070 0x7fc8a414e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8aa436640 1 -- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a40b7770 msgr2=0x7fc8a414f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.424+0000 7fc8aa436640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a40b7770 0x7fc8a414f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc8aa436640 1 -- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a4073070 msgr2=0x7fc8a414e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc8aa436640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a4073070 0x7fc8a414e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc8aa436640 1 -- 172.21.15.17:0/2972704860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a414e6e0 con 0x7fc8a4145740 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc8a9c35640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a4073070 0x7fc8a414e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc8aa436640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8a4145740 0x7fc8a414eae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00c6f0 tx=0x7fc89c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc89a7fc640 1 -- 172.21.15.17:0/2972704860 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c017070 con 0x7fc8a4145740 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc89a7fc640 1 -- 172.21.15.17:0/2972704860 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc89c005db0 con 0x7fc8a4145740 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc8abe99640 1 -- 172.21.15.17:0/2972704860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a41e0520 con 0x7fc8a4145740 2024-03-16T10:11:18.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc89a7fc640 1 -- 172.21.15.17:0/2972704860 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c00a840 con 0x7fc8a4145740 2024-03-16T10:11:18.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc8abe99640 1 -- 172.21.15.17:0/2972704860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a41e0ae0 con 0x7fc8a4145740 2024-03-16T10:11:18.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc8abe99640 1 -- 172.21.15.17:0/2972704860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a406a130 con 0x7fc8a4145740 2024-03-16T10:11:18.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc89a7fc640 1 -- 172.21.15.17:0/2972704860 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fc89c01a070 con 0x7fc8a4145740 2024-03-16T10:11:18.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc89a7fc640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc888067660 0x7fc888069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc89a7fc640 1 -- 172.21.15.17:0/2972704860 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc89c084c90 con 0x7fc8a4145740 2024-03-16T10:11:18.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc8a9c35640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc888067660 0x7fc888069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.428+0000 7fc8a9c35640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc888067660 0x7fc888069b20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009bc0 tx=0x7fc8a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:18.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.436+0000 7fc89a7fc640 1 -- 172.21.15.17:0/2972704860 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc89c0552d0 con 0x7fc8a4145740 2024-03-16T10:11:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.588+0000 7fc8abe99640 1 -- 172.21.15.17:0/2972704860 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc8a41e0330 con 0x7fc8a4145740 2024-03-16T10:11:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.588+0000 7fc89a7fc640 1 -- 172.21.15.17:0/2972704860 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fc89c059370 con 0x7fc8a4145740 2024-03-16T10:11:18.594 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-16T10:11:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.592+0000 7fc8abe99640 1 -- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc888067660 msgr2=0x7fc888069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.592+0000 7fc8abe99640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc888067660 0x7fc888069b20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0009bc0 tx=0x7fc8a0008040 comp rx=0 tx=0).stop 2024-03-16T10:11:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.592+0000 7fc8abe99640 1 -- 172.21.15.17:0/2972704860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8a4145740 msgr2=0x7fc8a414eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.592+0000 7fc8abe99640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8a4145740 0x7fc8a414eae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00c6f0 tx=0x7fc89c00cbb0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.592+0000 7fc8abe99640 1 -- 172.21.15.17:0/2972704860 shutdown_connections 2024-03-16T10:11:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.592+0000 7fc8abe99640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc888067660 0x7fc888069b20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.592+0000 7fc8abe99640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc8a4145740 0x7fc8a414eae0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.592+0000 7fc8abe99640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc8a40b7770 0x7fc8a414f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.592+0000 7fc8abe99640 1 --2- 172.21.15.17:0/2972704860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8a4073070 0x7fc8a414e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.592+0000 7fc8abe99640 1 -- 172.21.15.17:0/2972704860 >> 172.21.15.17:0/2972704860 conn(0x7fc8a406b290 msgr2=0x7fc8a40bb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.592+0000 7fc8abe99640 1 -- 172.21.15.17:0/2972704860 shutdown_connections 2024-03-16T10:11:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.592+0000 7fc8abe99640 1 -- 172.21.15.17:0/2972704860 wait complete. 2024-03-16T10:11:18.611 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:11:18.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.728+0000 7efcfd82b640 1 Processor -- start 2024-03-16T10:11:18.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.728+0000 7efcfd82b640 1 -- start start 2024-03-16T10:11:18.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.732+0000 7efcfd82b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcf80b5740 0x7efcf80aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.732+0000 7efcfd82b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcf806ede0 0x7efcf8071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.732+0000 7efcfd82b640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcf80b5b20 con 0x7efcf8072410 2024-03-16T10:11:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.732+0000 7efcfd82b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7efcf806e950 con 0x7efcf80b5740 2024-03-16T10:11:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.732+0000 7efcfd82b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7efcf80af720 con 0x7efcf806ede0 2024-03-16T10:11:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.732+0000 7efcf6ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcf80b5740 0x7efcf80aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.732+0000 7efcf6ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7efcf80b5740 0x7efcf80aeaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55818/0 (socket says 172.21.15.17:55818) 2024-03-16T10:11:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.732+0000 7efcf6ffd640 1 -- 172.21.15.17:0/1184747437 learned_addr learned my addr 172.21.15.17:0/1184747437 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.732+0000 7efcf67fc640 1 --2- 172.21.15.17:0/1184747437 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcf806ede0 0x7efcf8071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.732+0000 7efcf6ffd640 1 -- 172.21.15.17:0/1184747437 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcf806ede0 msgr2=0x7efcf8071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.732+0000 7efcf6ffd640 1 --2- 172.21.15.17:0/1184747437 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcf806ede0 0x7efcf8071ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.732+0000 7efcf6ffd640 1 -- 172.21.15.17:0/1184747437 >> v1:172.21.15.174:6789/0 conn(0x7efcf8072410 legacy=0x7efcf81440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.732+0000 7efcf6ffd640 1 -- 172.21.15.17:0/1184747437 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf80afa00 con 0x7efcf80b5740 2024-03-16T10:11:18.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.732+0000 7efcf67fc640 1 --2- 172.21.15.17:0/1184747437 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcf806ede0 0x7efcf8071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:18.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf6ffd640 1 --2- 172.21.15.17:0/1184747437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf80b5740 0x7efcf80aeaf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009870 tx=0x7efcf001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f10c3866f0d627 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:18.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf5ffb640 1 -- 172.21.15.17:0/1184747437 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00ab5f0 con 0x7efcf80b5740 2024-03-16T10:11:18.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf5ffb640 1 -- 172.21.15.17:0/1184747437 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf00b0070 con 0x7efcf80b5740 2024-03-16T10:11:18.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf5ffb640 1 -- 172.21.15.17:0/1184747437 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00ab9b0 con 0x7efcf80b5740 2024-03-16T10:11:18.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 -- 172.21.15.17:0/1184747437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf80b5740 msgr2=0x7efcf80aeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 --2- 172.21.15.17:0/1184747437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf80b5740 0x7efcf80aeaf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009870 tx=0x7efcf001c520 comp rx=0 tx=0).stop 2024-03-16T10:11:18.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 -- 172.21.15.17:0/1184747437 shutdown_connections 2024-03-16T10:11:18.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 --2- 172.21.15.17:0/1184747437 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcf806ede0 0x7efcf8071ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 --2- 172.21.15.17:0/1184747437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf80b5740 0x7efcf80aeaf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 -- 172.21.15.17:0/1184747437 >> 172.21.15.17:0/1184747437 conn(0x7efcf80b65d0 msgr2=0x7efcf80b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:18.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 -- 172.21.15.17:0/1184747437 shutdown_connections 2024-03-16T10:11:18.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 -- 172.21.15.17:0/1184747437 wait complete. 2024-03-16T10:11:18.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 Processor -- start 2024-03-16T10:11:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 -- start start 2024-03-16T10:11:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf806ede0 0x7efcf8141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcf8072410 0x7efcf813f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcf80b5740 0x7efcf813c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efcf806f1c0 con 0x7efcf806ede0 2024-03-16T10:11:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efcf806f340 con 0x7efcf8072410 2024-03-16T10:11:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7efcf81519d0 con 0x7efcf80b5740 2024-03-16T10:11:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf806ede0 0x7efcf8141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf806ede0 0x7efcf8141640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55824/0 (socket says 172.21.15.17:55824) 2024-03-16T10:11:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf6ffd640 1 -- 172.21.15.17:0/2787587558 learned_addr learned my addr 172.21.15.17:0/2787587558 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:18.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf67fc640 1 --2- 172.21.15.17:0/2787587558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcf8072410 0x7efcf813f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf6ffd640 1 -- 172.21.15.17:0/2787587558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcf80b5740 msgr2=0x7efcf813c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf77fe640 1 --2- 172.21.15.17:0/2787587558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcf80b5740 0x7efcf813c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf6ffd640 1 --2- 172.21.15.17:0/2787587558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcf80b5740 0x7efcf813c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf6ffd640 1 -- 172.21.15.17:0/2787587558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcf8072410 msgr2=0x7efcf813f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf6ffd640 1 --2- 172.21.15.17:0/2787587558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcf8072410 0x7efcf813f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf6ffd640 1 -- 172.21.15.17:0/2787587558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf8140970 con 0x7efcf806ede0 2024-03-16T10:11:18.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcf6ffd640 1 --2- 172.21.15.17:0/2787587558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf806ede0 0x7efcf8141640 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efcf00ad040 tx=0x7efcf001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:18.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efceffff640 1 -- 172.21.15.17:0/2787587558 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00be5f0 con 0x7efcf806ede0 2024-03-16T10:11:18.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efceffff640 1 -- 172.21.15.17:0/2787587558 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf00be790 con 0x7efcf806ede0 2024-03-16T10:11:18.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efcfd82b640 1 -- 172.21.15.17:0/2787587558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf813f7d0 con 0x7efcf806ede0 2024-03-16T10:11:18.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.736+0000 7efceffff640 1 -- 172.21.15.17:0/2787587558 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00beb60 con 0x7efcf806ede0 2024-03-16T10:11:18.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.740+0000 7efcfd82b640 1 -- 172.21.15.17:0/2787587558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf8140ec0 con 0x7efcf806ede0 2024-03-16T10:11:18.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.740+0000 7efcfd82b640 1 -- 172.21.15.17:0/2787587558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf813e100 con 0x7efcf806ede0 2024-03-16T10:11:18.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.740+0000 7efceffff640 1 -- 172.21.15.17:0/2787587558 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7efcf00b43f0 con 0x7efcf806ede0 2024-03-16T10:11:18.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.740+0000 7efceffff640 1 --2- 172.21.15.17:0/2787587558 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efcd8067660 0x7efcd8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:18.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.740+0000 7efceffff640 1 -- 172.21.15.17:0/2787587558 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efcf01201a0 con 0x7efcf806ede0 2024-03-16T10:11:18.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.740+0000 7efcf67fc640 1 --2- 172.21.15.17:0/2787587558 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efcd8067660 0x7efcd8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:18.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.740+0000 7efcf67fc640 1 --2- 172.21.15.17:0/2787587558 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efcd8067660 0x7efcd8069b20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efce0006fd0 tx=0x7efce0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:18.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.744+0000 7efceffff640 1 -- 172.21.15.17:0/2787587558 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7efcf00c4060 con 0x7efcf806ede0 2024-03-16T10:11:18.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.900+0000 7efcfd82b640 1 -- 172.21.15.17:0/2787587558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efcf806a130 con 0x7efcf806ede0 2024-03-16T10:11:18.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.900+0000 7efceffff640 1 -- 172.21.15.17:0/2787587558 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7efcf0016110 con 0x7efcf806ede0 2024-03-16T10:11:18.905 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-16T10:11:18.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.904+0000 7efcfd82b640 1 -- 172.21.15.17:0/2787587558 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efcd8067660 msgr2=0x7efcd8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.904+0000 7efcfd82b640 1 --2- 172.21.15.17:0/2787587558 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efcd8067660 0x7efcd8069b20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efce0006fd0 tx=0x7efce0008040 comp rx=0 tx=0).stop 2024-03-16T10:11:18.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.904+0000 7efcfd82b640 1 -- 172.21.15.17:0/2787587558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf806ede0 msgr2=0x7efcf8141640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:18.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.904+0000 7efcfd82b640 1 --2- 172.21.15.17:0/2787587558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf806ede0 0x7efcf8141640 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7efcf00ad040 tx=0x7efcf001ad60 comp rx=0 tx=0).stop 2024-03-16T10:11:18.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.904+0000 7efcfd82b640 1 -- 172.21.15.17:0/2787587558 shutdown_connections 2024-03-16T10:11:18.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.904+0000 7efcfd82b640 1 --2- 172.21.15.17:0/2787587558 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efcd8067660 0x7efcd8069b20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.904+0000 7efcfd82b640 1 --2- 172.21.15.17:0/2787587558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efcf80b5740 0x7efcf813c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.904+0000 7efcfd82b640 1 --2- 172.21.15.17:0/2787587558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcf8072410 0x7efcf813f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.904+0000 7efcfd82b640 1 --2- 172.21.15.17:0/2787587558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcf806ede0 0x7efcf8141640 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:18.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.904+0000 7efcfd82b640 1 -- 172.21.15.17:0/2787587558 >> 172.21.15.17:0/2787587558 conn(0x7efcf80b65d0 msgr2=0x7efcf806bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:18.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.904+0000 7efcfd82b640 1 -- 172.21.15.17:0/2787587558 shutdown_connections 2024-03-16T10:11:18.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:18.904+0000 7efcfd82b640 1 -- 172.21.15.17:0/2787587558 wait complete. 2024-03-16T10:11:18.970 INFO:tasks.cephfs_test_runner:Starting test: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) Test that when cd is issued with an argument, CWD is changed 2024-03-16T10:11:18.970 INFO:tasks.cephfs_test_runner:test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-16T10:11:18.970 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:19.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff604e69640 1 Processor -- start 2024-03-16T10:11:19.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff604e69640 1 -- start start 2024-03-16T10:11:19.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff604e69640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff60006e210 0x7ff6000b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff604e69640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff60006ee90 0x7ff60009ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff604e69640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff60006e5f0 con 0x7ff60009efc0 2024-03-16T10:11:19.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff604e69640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ff6000b0680 con 0x7ff60006e210 2024-03-16T10:11:19.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff604e69640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff6000af160 con 0x7ff60006ee90 2024-03-16T10:11:19.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fed9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7ff60009efc0 0x7ff600144400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46104/0 (socket says 172.21.15.17:46104) 2024-03-16T10:11:19.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fed9d640 1 -- 172.21.15.17:0/2033014558 learned_addr learned my addr 172.21.15.17:0/2033014558 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:19.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fe59c640 1 --2- 172.21.15.17:0/2033014558 >> v2:172.21.15.174:3300/0 conn(0x7ff60006e210 0x7ff6000b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fdd9b640 1 --2- 172.21.15.17:0/2033014558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff60006ee90 0x7ff60009ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 -- 172.21.15.17:0/2033014558 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2205968478 0 0) 0x7ff60006e5f0 con 0x7ff60009efc0 2024-03-16T10:11:19.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 -- 172.21.15.17:0/2033014558 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5ec003530 con 0x7ff60009efc0 2024-03-16T10:11:19.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 -- 172.21.15.17:0/2033014558 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 218913761 0 0) 0x7ff5ec003530 con 0x7ff60009efc0 2024-03-16T10:11:19.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 -- 172.21.15.17:0/2033014558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff60006ee90 msgr2=0x7ff60009ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:19.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 --2- 172.21.15.17:0/2033014558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff60006ee90 0x7ff60009ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 -- 172.21.15.17:0/2033014558 >> v2:172.21.15.174:3300/0 conn(0x7ff60006e210 msgr2=0x7ff6000b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:19.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 --2- 172.21.15.17:0/2033014558 >> v2:172.21.15.174:3300/0 conn(0x7ff60006e210 0x7ff6000b0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 -- 172.21.15.17:0/2033014558 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6000af440 con 0x7ff60009efc0 2024-03-16T10:11:19.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fe59c640 1 --2- 172.21.15.17:0/2033014558 >> v2:172.21.15.174:3300/0 conn(0x7ff60006e210 0x7ff6000b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:11:19.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 -- 172.21.15.17:0/2033014558 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7ff5f8001a00 con 0x7ff60009efc0 2024-03-16T10:11:19.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 -- 172.21.15.17:0/2033014558 >> v1:172.21.15.17:6789/0 conn(0x7ff60009efc0 legacy=0x7ff600144400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 --2- 172.21.15.17:0/2033014558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec0039c0 0x7ff5ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 --2- 172.21.15.17:0/2033014558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec004810 0x7ff5ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 --2- 172.21.15.17:0/2033014558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff60006ee90 0x7ff5ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 -- 172.21.15.17:0/2033014558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff6000b0680 con 0x7ff5ec0039c0 2024-03-16T10:11:19.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 -- 172.21.15.17:0/2033014558 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff6000af160 con 0x7ff5ec004810 2024-03-16T10:11:19.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fd59a640 1 -- 172.21.15.17:0/2033014558 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ec00dc60 con 0x7ff60006ee90 2024-03-16T10:11:19.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fe59c640 1 --2- 172.21.15.17:0/2033014558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec0039c0 0x7ff5ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fdd9b640 1 --2- 172.21.15.17:0/2033014558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec004810 0x7ff5ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fe59c640 1 -- 172.21.15.17:0/2033014558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff60006ee90 msgr2=0x7ff5ec0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:19.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fe59c640 1 --2- 172.21.15.17:0/2033014558 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff60006ee90 0x7ff5ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fe59c640 1 -- 172.21.15.17:0/2033014558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec004810 msgr2=0x7ff5ec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:19.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fe59c640 1 --2- 172.21.15.17:0/2033014558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec004810 0x7ff5ec004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.100+0000 7ff5fe59c640 1 -- 172.21.15.17:0/2033014558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec003530 con 0x7ff5ec0039c0 2024-03-16T10:11:19.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff5fe59c640 1 --2- 172.21.15.17:0/2033014558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec0039c0 0x7ff5ec003de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009cb0 tx=0x7ff5f001cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde90cbca6e63c38 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:19.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff5fd59a640 1 -- 172.21.15.17:0/2033014558 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0009880 con 0x7ff5ec0039c0 2024-03-16T10:11:19.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff5fd59a640 1 -- 172.21.15.17:0/2033014558 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f00a65c0 con 0x7ff5ec0039c0 2024-03-16T10:11:19.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff5fd59a640 1 -- 172.21.15.17:0/2033014558 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f00a69b0 con 0x7ff5ec0039c0 2024-03-16T10:11:19.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff604e69640 1 -- 172.21.15.17:0/2033014558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec0039c0 msgr2=0x7ff5ec003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff604e69640 1 --2- 172.21.15.17:0/2033014558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5ec0039c0 0x7ff5ec003de0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff5f0009cb0 tx=0x7ff5f001cc90 comp rx=0 tx=0).stop 2024-03-16T10:11:19.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff5fed9d640 1 -- 172.21.15.17:0/2033014558 reap_dead start 2024-03-16T10:11:19.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff604e69640 1 -- 172.21.15.17:0/2033014558 shutdown_connections 2024-03-16T10:11:19.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff604e69640 1 -- 172.21.15.17:0/2033014558 >> 172.21.15.17:0/2033014558 conn(0x7ff6000ba0e0 msgr2=0x7ff6000ba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:19.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff604e69640 1 -- 172.21.15.17:0/2033014558 shutdown_connections 2024-03-16T10:11:19.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff604e69640 1 -- 172.21.15.17:0/2033014558 wait complete. 2024-03-16T10:11:19.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff604e69640 1 Processor -- start 2024-03-16T10:11:19.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff604e69640 1 -- start start 2024-03-16T10:11:19.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff604e69640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec0039c0 0x7ff600071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff604e69640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60014a500 0x7ff6001468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff604e69640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff600145c20 0x7ff600148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff604e69640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff60006be30 con 0x7ff60014a500 2024-03-16T10:11:19.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff604e69640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff60006bcb0 con 0x7ff5ec0039c0 2024-03-16T10:11:19.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff604e69640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff60006bfb0 con 0x7ff600145c20 2024-03-16T10:11:19.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff5fdd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60014a500 0x7ff6001468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff5fdd9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60014a500 0x7ff6001468d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55178/0 (socket says 172.21.15.17:55178) 2024-03-16T10:11:19.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.104+0000 7ff5fdd9b640 1 -- 172.21.15.17:0/484560251 learned_addr learned my addr 172.21.15.17:0/484560251 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:19.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5fed9d640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff600145c20 0x7ff600148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5fe59c640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec0039c0 0x7ff600071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5fdd9b640 1 -- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff600145c20 msgr2=0x7ff600148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5fdd9b640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff600145c20 0x7ff600148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5fdd9b640 1 -- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec0039c0 msgr2=0x7ff600071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5fdd9b640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec0039c0 0x7ff600071ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5fdd9b640 1 -- 172.21.15.17:0/484560251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff600148440 con 0x7ff60014a500 2024-03-16T10:11:19.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5fed9d640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff600145c20 0x7ff600148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:19.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5fe59c640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec0039c0 0x7ff600071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:19.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5fdd9b640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60014a500 0x7ff6001468d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80108c0 tx=0x7ff5e8010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:19.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5f6ffd640 1 -- 172.21.15.17:0/484560251 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e800f040 con 0x7ff60014a500 2024-03-16T10:11:19.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5f6ffd640 1 -- 172.21.15.17:0/484560251 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e800eb60 con 0x7ff60014a500 2024-03-16T10:11:19.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff604e69640 1 -- 172.21.15.17:0/484560251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff600145940 con 0x7ff60014a500 2024-03-16T10:11:19.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5f6ffd640 1 -- 172.21.15.17:0/484560251 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e801c400 con 0x7ff60014a500 2024-03-16T10:11:19.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff604e69640 1 -- 172.21.15.17:0/484560251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff600146e10 con 0x7ff60014a500 2024-03-16T10:11:19.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff604e69640 1 -- 172.21.15.17:0/484560251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60006a130 con 0x7ff60014a500 2024-03-16T10:11:19.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5f6ffd640 1 -- 172.21.15.17:0/484560251 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7ff5e801c5a0 con 0x7ff60014a500 2024-03-16T10:11:19.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5f6ffd640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff5e0067690 0x7ff5e0069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5f6ffd640 1 -- 172.21.15.17:0/484560251 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff5e8085eb0 con 0x7ff60014a500 2024-03-16T10:11:19.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5fe59c640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff5e0067690 0x7ff5e0069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.108+0000 7ff5fe59c640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff5e0067690 0x7ff5e0069b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00af000 tx=0x7ff5f00046d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:19.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.112+0000 7ff5f6ffd640 1 -- 172.21.15.17:0/484560251 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff5e8018030 con 0x7ff60014a500 2024-03-16T10:11:19.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.272+0000 7ff604e69640 1 -- 172.21.15.17:0/484560251 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6000b7ea0 con 0x7ff60014a500 2024-03-16T10:11:19.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.272+0000 7ff5f6ffd640 1 -- 172.21.15.17:0/484560251 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7ff5e8056b40 con 0x7ff60014a500 2024-03-16T10:11:19.278 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:19.278 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:19.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.276+0000 7ff604e69640 1 -- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff5e0067690 msgr2=0x7ff5e0069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.276+0000 7ff604e69640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff5e0067690 0x7ff5e0069b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00af000 tx=0x7ff5f00046d0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.276+0000 7ff604e69640 1 -- 172.21.15.17:0/484560251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60014a500 msgr2=0x7ff6001468d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.276+0000 7ff604e69640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60014a500 0x7ff6001468d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80108c0 tx=0x7ff5e8010d80 comp rx=0 tx=0).stop 2024-03-16T10:11:19.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.276+0000 7ff604e69640 1 -- 172.21.15.17:0/484560251 shutdown_connections 2024-03-16T10:11:19.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.276+0000 7ff604e69640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff5e0067690 0x7ff5e0069b50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.276+0000 7ff604e69640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff600145c20 0x7ff600148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.276+0000 7ff604e69640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff60014a500 0x7ff6001468d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.276+0000 7ff604e69640 1 --2- 172.21.15.17:0/484560251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5ec0039c0 0x7ff600071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.276+0000 7ff604e69640 1 -- 172.21.15.17:0/484560251 >> 172.21.15.17:0/484560251 conn(0x7ff6000ba0e0 msgr2=0x7ff6000b4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:19.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.276+0000 7ff604e69640 1 -- 172.21.15.17:0/484560251 shutdown_connections 2024-03-16T10:11:19.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.276+0000 7ff604e69640 1 -- 172.21.15.17:0/484560251 wait complete. 2024-03-16T10:11:19.296 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:10:55.723734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:10:56.459894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:11:19.296 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:11:19.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f365f073640 1 Processor -- start 2024-03-16T10:11:19.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f365f073640 1 -- start start 2024-03-16T10:11:19.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f365f073640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f36580b54e0 0x7f365806e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f365f073640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f365813f090 0x7f365813f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f365f073640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f365806f220 con 0x7f365806ee40 2024-03-16T10:11:19.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f365f073640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f36580b5970 con 0x7f36580b54e0 2024-03-16T10:11:19.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f365f073640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f36580b5c50 con 0x7f365813f090 2024-03-16T10:11:19.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f3657fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f36580b54e0 0x7f365806e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f3657fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f36580b54e0 0x7f365806e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55186/0 (socket says 172.21.15.17:55186) 2024-03-16T10:11:19.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f3657fff640 1 -- 172.21.15.17:0/4000224721 learned_addr learned my addr 172.21.15.17:0/4000224721 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:19.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f365d610640 1 --2- 172.21.15.17:0/4000224721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f365813f090 0x7f365813f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f3657fff640 1 -- 172.21.15.17:0/4000224721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f365813f090 msgr2=0x7f365813f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:19.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f3657fff640 1 --2- 172.21.15.17:0/4000224721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f365813f090 0x7f365813f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f3657fff640 1 -- 172.21.15.17:0/4000224721 >> v1:172.21.15.174:6789/0 conn(0x7f365806ee40 legacy=0x7f3658071f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.404+0000 7f3657fff640 1 -- 172.21.15.17:0/4000224721 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3658071a00 con 0x7f36580b54e0 2024-03-16T10:11:19.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f3657fff640 1 --2- 172.21.15.17:0/4000224721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36580b54e0 0x7f365806e3e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3644009870 tx=0x7f364401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fd47f3370b2ec1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:19.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f36577fe640 1 -- 172.21.15.17:0/4000224721 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36440b0070 con 0x7f36580b54e0 2024-03-16T10:11:19.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f36577fe640 1 -- 172.21.15.17:0/4000224721 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36440ab600 con 0x7f36580b54e0 2024-03-16T10:11:19.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f36577fe640 1 -- 172.21.15.17:0/4000224721 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36440ab9d0 con 0x7f36580b54e0 2024-03-16T10:11:19.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 -- 172.21.15.17:0/4000224721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36580b54e0 msgr2=0x7f365806e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 --2- 172.21.15.17:0/4000224721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36580b54e0 0x7f365806e3e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3644009870 tx=0x7f364401c4d0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 -- 172.21.15.17:0/4000224721 shutdown_connections 2024-03-16T10:11:19.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 --2- 172.21.15.17:0/4000224721 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f365813f090 0x7f365813f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 --2- 172.21.15.17:0/4000224721 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f36580b54e0 0x7f365806e3e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 -- 172.21.15.17:0/4000224721 >> 172.21.15.17:0/4000224721 conn(0x7f36580b65d0 msgr2=0x7f36580b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:19.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 -- 172.21.15.17:0/4000224721 shutdown_connections 2024-03-16T10:11:19.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 -- 172.21.15.17:0/4000224721 wait complete. 2024-03-16T10:11:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 Processor -- start 2024-03-16T10:11:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 -- start start 2024-03-16T10:11:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f365806ee40 0x7f3658136220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f365813f090 0x7f365809f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36580b0730 0x7f3658137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f36581526c0 con 0x7f365813f090 2024-03-16T10:11:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3658152540 con 0x7f36580b0730 2024-03-16T10:11:19.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365f073640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3658152840 con 0x7f365806ee40 2024-03-16T10:11:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f3657fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f365813f090 0x7f365809f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f3657fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f365813f090 0x7f365809f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55202/0 (socket says 172.21.15.17:55202) 2024-03-16T10:11:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f3657fff640 1 -- 172.21.15.17:0/2354668654 learned_addr learned my addr 172.21.15.17:0/2354668654 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365ce0f640 1 --2- 172.21.15.17:0/2354668654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f365806ee40 0x7f3658136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f365d610640 1 --2- 172.21.15.17:0/2354668654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36580b0730 0x7f3658137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f3657fff640 1 -- 172.21.15.17:0/2354668654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f365806ee40 msgr2=0x7f3658136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f3657fff640 1 --2- 172.21.15.17:0/2354668654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f365806ee40 0x7f3658136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f3657fff640 1 -- 172.21.15.17:0/2354668654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36580b0730 msgr2=0x7f3658137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f3657fff640 1 --2- 172.21.15.17:0/2354668654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36580b0730 0x7f3658137b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f3657fff640 1 -- 172.21.15.17:0/2354668654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3658135d70 con 0x7f365813f090 2024-03-16T10:11:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.408+0000 7f3657fff640 1 --2- 172.21.15.17:0/2354668654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f365813f090 0x7f365809f0b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f36440ae040 tx=0x7f3644019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.412+0000 7f36557fa640 1 -- 172.21.15.17:0/2354668654 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36440b0040 con 0x7f365813f090 2024-03-16T10:11:19.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.412+0000 7f36557fa640 1 -- 172.21.15.17:0/2354668654 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36440b9070 con 0x7f365813f090 2024-03-16T10:11:19.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.412+0000 7f365f073640 1 -- 172.21.15.17:0/2354668654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36580af9a0 con 0x7f365813f090 2024-03-16T10:11:19.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.412+0000 7f36557fa640 1 -- 172.21.15.17:0/2354668654 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36440b4660 con 0x7f365813f090 2024-03-16T10:11:19.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.412+0000 7f365f073640 1 -- 172.21.15.17:0/2354668654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36580afc80 con 0x7f365813f090 2024-03-16T10:11:19.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.412+0000 7f365f073640 1 -- 172.21.15.17:0/2354668654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f365806a130 con 0x7f365813f090 2024-03-16T10:11:19.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.412+0000 7f36557fa640 1 -- 172.21.15.17:0/2354668654 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f36440b4800 con 0x7f365813f090 2024-03-16T10:11:19.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.412+0000 7f36557fa640 1 --2- 172.21.15.17:0/2354668654 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f363c067660 0x7f363c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.412+0000 7f36557fa640 1 -- 172.21.15.17:0/2354668654 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f364411f510 con 0x7f365813f090 2024-03-16T10:11:19.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.412+0000 7f365ce0f640 1 --2- 172.21.15.17:0/2354668654 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f363c067660 0x7f363c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.412+0000 7f365ce0f640 1 --2- 172.21.15.17:0/2354668654 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f363c067660 0x7f363c069b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f36581386b0 tx=0x7f364c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:19.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.416+0000 7f36557fa640 1 -- 172.21.15.17:0/2354668654 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3644016150 con 0x7f365813f090 2024-03-16T10:11:19.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.572+0000 7f365f073640 1 -- 172.21.15.17:0/2354668654 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3658143d30 con 0x7f365813f090 2024-03-16T10:11:19.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.572+0000 7f36557fa640 1 -- 172.21.15.17:0/2354668654 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f36440f0220 con 0x7f365813f090 2024-03-16T10:11:19.577 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:11:19.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.572+0000 7f365f073640 1 -- 172.21.15.17:0/2354668654 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f363c067660 msgr2=0x7f363c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.572+0000 7f365f073640 1 --2- 172.21.15.17:0/2354668654 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f363c067660 0x7f363c069b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f36581386b0 tx=0x7f364c009040 comp rx=0 tx=0).stop 2024-03-16T10:11:19.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.572+0000 7f365f073640 1 -- 172.21.15.17:0/2354668654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f365813f090 msgr2=0x7f365809f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.572+0000 7f365f073640 1 --2- 172.21.15.17:0/2354668654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f365813f090 0x7f365809f0b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f36440ae040 tx=0x7f3644019270 comp rx=0 tx=0).stop 2024-03-16T10:11:19.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.572+0000 7f365f073640 1 -- 172.21.15.17:0/2354668654 shutdown_connections 2024-03-16T10:11:19.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.572+0000 7f365f073640 1 --2- 172.21.15.17:0/2354668654 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f363c067660 0x7f363c069b20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.572+0000 7f365f073640 1 --2- 172.21.15.17:0/2354668654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36580b0730 0x7f3658137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.572+0000 7f365f073640 1 --2- 172.21.15.17:0/2354668654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f365813f090 0x7f365809f0b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.572+0000 7f365f073640 1 --2- 172.21.15.17:0/2354668654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f365806ee40 0x7f3658136220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.572+0000 7f365f073640 1 -- 172.21.15.17:0/2354668654 >> 172.21.15.17:0/2354668654 conn(0x7f36580b65d0 msgr2=0x7f365806b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:19.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.572+0000 7f365f073640 1 -- 172.21.15.17:0/2354668654 shutdown_connections 2024-03-16T10:11:19.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.572+0000 7f365f073640 1 -- 172.21.15.17:0/2354668654 wait complete. 2024-03-16T10:11:19.594 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:11:19.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.704+0000 7f3a27f57640 1 Processor -- start 2024-03-16T10:11:19.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.704+0000 7f3a27f57640 1 -- start start 2024-03-16T10:11:19.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a27f57640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a200b5740 0x7f3a200aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a27f57640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3a2006ede0 0x7f3a20071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a27f57640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a200b5b20 con 0x7f3a20072410 2024-03-16T10:11:19.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a27f57640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f3a2006e950 con 0x7f3a2006ede0 2024-03-16T10:11:19.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a27f57640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3a200af720 con 0x7f3a200b5740 2024-03-16T10:11:19.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a264f4640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f3a20072410 0x7f3a201440b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46106/0 (socket says 172.21.15.17:46106) 2024-03-16T10:11:19.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a264f4640 1 -- 172.21.15.17:0/78332428 learned_addr learned my addr 172.21.15.17:0/78332428 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:19.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a254f2640 1 --2- 172.21.15.17:0/78332428 >> v2:172.21.15.174:3300/0 conn(0x7f3a2006ede0 0x7f3a20071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a25cf3640 1 --2- 172.21.15.17:0/78332428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a200b5740 0x7f3a200aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 -- 172.21.15.17:0/78332428 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3012399326 0 0) 0x7f3a200b5b20 con 0x7f3a20072410 2024-03-16T10:11:19.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 -- 172.21.15.17:0/78332428 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a0c003530 con 0x7f3a20072410 2024-03-16T10:11:19.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 -- 172.21.15.17:0/78332428 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1995134785 0 0) 0x7f3a0c003530 con 0x7f3a20072410 2024-03-16T10:11:19.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 -- 172.21.15.17:0/78332428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a200b5740 msgr2=0x7f3a200aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:19.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 --2- 172.21.15.17:0/78332428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a200b5740 0x7f3a200aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 -- 172.21.15.17:0/78332428 >> v2:172.21.15.174:3300/0 conn(0x7f3a2006ede0 msgr2=0x7f3a20071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:19.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 --2- 172.21.15.17:0/78332428 >> v2:172.21.15.174:3300/0 conn(0x7f3a2006ede0 0x7f3a20071ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 -- 172.21.15.17:0/78332428 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a200afa00 con 0x7f3a20072410 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 -- 172.21.15.17:0/78332428 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f3a18001680 con 0x7f3a20072410 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 -- 172.21.15.17:0/78332428 >> v1:172.21.15.17:6789/0 conn(0x7f3a20072410 legacy=0x7f3a201440b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 --2- 172.21.15.17:0/78332428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a200b5740 0x7f3a0c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 --2- 172.21.15.17:0/78332428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a0c003f40 0x7f3a0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 --2- 172.21.15.17:0/78332428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a0c004d50 0x7f3a0c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 -- 172.21.15.17:0/78332428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2006e950 con 0x7f3a0c004d50 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 -- 172.21.15.17:0/78332428 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a200af720 con 0x7f3a0c003f40 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a24cf1640 1 -- 172.21.15.17:0/78332428 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3a0c00dbe0 con 0x7f3a200b5740 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a264f4640 1 --2- 172.21.15.17:0/78332428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a0c004d50 0x7f3a0c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a254f2640 1 --2- 172.21.15.17:0/78332428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a0c003f40 0x7f3a0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a25cf3640 1 --2- 172.21.15.17:0/78332428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a200b5740 0x7f3a0c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a264f4640 1 -- 172.21.15.17:0/78332428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a200b5740 msgr2=0x7f3a0c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a264f4640 1 --2- 172.21.15.17:0/78332428 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a200b5740 0x7f3a0c0039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a264f4640 1 -- 172.21.15.17:0/78332428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a0c003f40 msgr2=0x7f3a0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:19.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a264f4640 1 --2- 172.21.15.17:0/78332428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a0c003f40 0x7f3a0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a264f4640 1 -- 172.21.15.17:0/78332428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0c003530 con 0x7f3a0c004d50 2024-03-16T10:11:19.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.708+0000 7f3a264f4640 1 --2- 172.21.15.17:0/78332428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a0c004d50 0x7f3a0c0051d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c2c0 tx=0x7f3a1801cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19dffbc798967c02 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:19.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a24cf1640 1 -- 172.21.15.17:0/78332428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18007d60 con 0x7f3a0c004d50 2024-03-16T10:11:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a24cf1640 1 -- 172.21.15.17:0/78332428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a180a9020 con 0x7f3a0c004d50 2024-03-16T10:11:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a24cf1640 1 -- 172.21.15.17:0/78332428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a180a5bd0 con 0x7f3a0c004d50 2024-03-16T10:11:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 -- 172.21.15.17:0/78332428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a0c004d50 msgr2=0x7f3a0c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 --2- 172.21.15.17:0/78332428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a0c004d50 0x7f3a0c0051d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c2c0 tx=0x7f3a1801cce0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a264f4640 1 -- 172.21.15.17:0/78332428 reap_dead start 2024-03-16T10:11:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 -- 172.21.15.17:0/78332428 shutdown_connections 2024-03-16T10:11:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 -- 172.21.15.17:0/78332428 >> 172.21.15.17:0/78332428 conn(0x7f3a200b65d0 msgr2=0x7f3a200b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 -- 172.21.15.17:0/78332428 shutdown_connections 2024-03-16T10:11:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 -- 172.21.15.17:0/78332428 wait complete. 2024-03-16T10:11:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 Processor -- start 2024-03-16T10:11:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 -- start start 2024-03-16T10:11:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a200b21c0 0x7f3a200b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a200b0f30 0x7f3a200b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a20139830 0x7f3a20139cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3a200af4e0 con 0x7f3a20139830 2024-03-16T10:11:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a200af360 con 0x7f3a200b0f30 2024-03-16T10:11:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3a2006f220 con 0x7f3a200b21c0 2024-03-16T10:11:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a264f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a20139830 0x7f3a20139cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a264f4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a20139830 0x7f3a20139cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55210/0 (socket says 172.21.15.17:55210) 2024-03-16T10:11:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a264f4640 1 -- 172.21.15.17:0/147627114 learned_addr learned my addr 172.21.15.17:0/147627114 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a25cf3640 1 --2- 172.21.15.17:0/147627114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a200b21c0 0x7f3a200b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a254f2640 1 --2- 172.21.15.17:0/147627114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a200b0f30 0x7f3a200b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a264f4640 1 -- 172.21.15.17:0/147627114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a200b21c0 msgr2=0x7f3a200b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a264f4640 1 --2- 172.21.15.17:0/147627114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a200b21c0 0x7f3a200b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a264f4640 1 -- 172.21.15.17:0/147627114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a200b0f30 msgr2=0x7f3a200b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a264f4640 1 --2- 172.21.15.17:0/147627114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a200b0f30 0x7f3a200b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a264f4640 1 -- 172.21.15.17:0/147627114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a200b18d0 con 0x7f3a20139830 2024-03-16T10:11:19.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a264f4640 1 --2- 172.21.15.17:0/147627114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a20139830 0x7f3a20139cb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3a1801cb10 tx=0x7f3a0c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:19.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a167fc640 1 -- 172.21.15.17:0/147627114 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18007d60 con 0x7f3a20139830 2024-03-16T10:11:19.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a167fc640 1 -- 172.21.15.17:0/147627114 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a180b2070 con 0x7f3a20139830 2024-03-16T10:11:19.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 -- 172.21.15.17:0/147627114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a201d8bd0 con 0x7f3a20139830 2024-03-16T10:11:19.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a167fc640 1 -- 172.21.15.17:0/147627114 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18001c90 con 0x7f3a20139830 2024-03-16T10:11:19.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.712+0000 7f3a27f57640 1 -- 172.21.15.17:0/147627114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a200b3980 con 0x7f3a20139830 2024-03-16T10:11:19.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.716+0000 7f3a27f57640 1 -- 172.21.15.17:0/147627114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a201d9290 con 0x7f3a20139830 2024-03-16T10:11:19.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.716+0000 7f3a167fc640 1 -- 172.21.15.17:0/147627114 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f3a18004cb0 con 0x7f3a20139830 2024-03-16T10:11:19.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.716+0000 7f3a167fc640 1 --2- 172.21.15.17:0/147627114 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3a04067660 0x7f3a04069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:19.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.716+0000 7f3a167fc640 1 -- 172.21.15.17:0/147627114 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3a1811e7a0 con 0x7f3a20139830 2024-03-16T10:11:19.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.716+0000 7f3a25cf3640 1 --2- 172.21.15.17:0/147627114 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3a04067660 0x7f3a04069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:19.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.716+0000 7f3a25cf3640 1 --2- 172.21.15.17:0/147627114 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3a04067660 0x7f3a04069b20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c003c10 tx=0x7f3a1c00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:19.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.720+0000 7f3a167fc640 1 -- 172.21.15.17:0/147627114 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3a201d9290 con 0x7f3a20139830 2024-03-16T10:11:19.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.880+0000 7f3a27f57640 1 -- 172.21.15.17:0/147627114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3a2006a130 con 0x7f3a20139830 2024-03-16T10:11:19.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.880+0000 7f3a167fc640 1 -- 172.21.15.17:0/147627114 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f3a18016a50 con 0x7f3a20139830 2024-03-16T10:11:19.886 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-16T10:11:19.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.884+0000 7f3a27f57640 1 -- 172.21.15.17:0/147627114 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3a04067660 msgr2=0x7f3a04069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.884+0000 7f3a27f57640 1 --2- 172.21.15.17:0/147627114 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3a04067660 0x7f3a04069b20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c003c10 tx=0x7f3a1c00b040 comp rx=0 tx=0).stop 2024-03-16T10:11:19.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.884+0000 7f3a27f57640 1 -- 172.21.15.17:0/147627114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a20139830 msgr2=0x7f3a20139cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:19.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.884+0000 7f3a27f57640 1 --2- 172.21.15.17:0/147627114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a20139830 0x7f3a20139cb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3a1801cb10 tx=0x7f3a0c005840 comp rx=0 tx=0).stop 2024-03-16T10:11:19.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.884+0000 7f3a27f57640 1 -- 172.21.15.17:0/147627114 shutdown_connections 2024-03-16T10:11:19.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.884+0000 7f3a27f57640 1 --2- 172.21.15.17:0/147627114 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3a04067660 0x7f3a04069b20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.884+0000 7f3a27f57640 1 --2- 172.21.15.17:0/147627114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a20139830 0x7f3a20139cb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.884+0000 7f3a27f57640 1 --2- 172.21.15.17:0/147627114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a200b0f30 0x7f3a200b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.884+0000 7f3a27f57640 1 --2- 172.21.15.17:0/147627114 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3a200b21c0 0x7f3a200b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:19.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.884+0000 7f3a27f57640 1 -- 172.21.15.17:0/147627114 >> 172.21.15.17:0/147627114 conn(0x7f3a200b65d0 msgr2=0x7f3a2013e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:19.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.884+0000 7f3a27f57640 1 -- 172.21.15.17:0/147627114 shutdown_connections 2024-03-16T10:11:19.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:19.884+0000 7f3a27f57640 1 -- 172.21.15.17:0/147627114 wait complete. 2024-03-16T10:11:19.904 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:11:20.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac36ae7640 1 Processor -- start 2024-03-16T10:11:20.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac36ae7640 1 -- start start 2024-03-16T10:11:20.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac36ae7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac300b4650 0x7fac3014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:20.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac36ae7640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fac301419f0 0x7fac30151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:20.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac36ae7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac300b4a30 con 0x7fac300b7650 2024-03-16T10:11:20.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac36ae7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fac3014abe0 con 0x7fac300b4650 2024-03-16T10:11:20.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac36ae7640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fac3014aec0 con 0x7fac301419f0 2024-03-16T10:11:20.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac34883640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac300b4650 0x7fac3014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:20.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac34883640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fac300b4650 0x7fac3014b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55222/0 (socket says 172.21.15.17:55222) 2024-03-16T10:11:20.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac2ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fac300b7650 0x7fac301412e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46116/0 (socket says 172.21.15.17:46116) 2024-03-16T10:11:20.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac34883640 1 -- 172.21.15.17:0/2108629271 learned_addr learned my addr 172.21.15.17:0/2108629271 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:20.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac35084640 1 --2- 172.21.15.17:0/2108629271 >> v2:172.21.15.174:3300/0 conn(0x7fac301419f0 0x7fac30151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:20.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac34883640 1 -- 172.21.15.17:0/2108629271 >> v2:172.21.15.174:3300/0 conn(0x7fac301419f0 msgr2=0x7fac30151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:20.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac34883640 1 --2- 172.21.15.17:0/2108629271 >> v2:172.21.15.174:3300/0 conn(0x7fac301419f0 0x7fac30151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:20.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac34883640 1 -- 172.21.15.17:0/2108629271 >> v1:172.21.15.17:6789/0 conn(0x7fac300b7650 legacy=0x7fac301412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:20.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac34883640 1 -- 172.21.15.17:0/2108629271 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3014b920 con 0x7fac300b4650 2024-03-16T10:11:20.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac2f7fe640 1 -- 172.21.15.17:0/2108629271 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3210074473 0 0) 0x7fac300b4a30 con 0x7fac300b7650 2024-03-16T10:11:20.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.024+0000 7fac35084640 1 --2- 172.21.15.17:0/2108629271 >> v2:172.21.15.174:3300/0 conn(0x7fac301419f0 0x7fac30151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:20.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.028+0000 7fac34883640 1 --2- 172.21.15.17:0/2108629271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac300b4650 0x7fac3014b3e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fac3006d9b0 tx=0x7fac2401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc66ea25bf94417e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:20.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.028+0000 7fac2f7fe640 1 -- 172.21.15.17:0/2108629271 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac240b0070 con 0x7fac300b4650 2024-03-16T10:11:20.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.028+0000 7fac2f7fe640 1 -- 172.21.15.17:0/2108629271 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac240ab600 con 0x7fac300b4650 2024-03-16T10:11:20.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.028+0000 7fac2f7fe640 1 -- 172.21.15.17:0/2108629271 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac240ab9d0 con 0x7fac300b4650 2024-03-16T10:11:20.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.028+0000 7fac36ae7640 1 -- 172.21.15.17:0/2108629271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac300b4650 msgr2=0x7fac3014b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:20.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.028+0000 7fac36ae7640 1 --2- 172.21.15.17:0/2108629271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac300b4650 0x7fac3014b3e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fac3006d9b0 tx=0x7fac2401c490 comp rx=0 tx=0).stop 2024-03-16T10:11:20.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.028+0000 7fac36ae7640 1 -- 172.21.15.17:0/2108629271 shutdown_connections 2024-03-16T10:11:20.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.028+0000 7fac36ae7640 1 --2- 172.21.15.17:0/2108629271 >> v2:172.21.15.174:3300/0 conn(0x7fac301419f0 0x7fac30151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:20.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.028+0000 7fac36ae7640 1 --2- 172.21.15.17:0/2108629271 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac300b4650 0x7fac3014b3e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:20.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.028+0000 7fac36ae7640 1 -- 172.21.15.17:0/2108629271 >> 172.21.15.17:0/2108629271 conn(0x7fac300bccd0 msgr2=0x7fac300bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:20.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.028+0000 7fac36ae7640 1 -- 172.21.15.17:0/2108629271 shutdown_connections 2024-03-16T10:11:20.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.028+0000 7fac36ae7640 1 -- 172.21.15.17:0/2108629271 wait complete. 2024-03-16T10:11:20.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.028+0000 7fac36ae7640 1 Processor -- start 2024-03-16T10:11:20.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.028+0000 7fac36ae7640 1 -- start start 2024-03-16T10:11:20.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac36ae7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac300b4650 0x7fac30144270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:20.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac36ae7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac300b7650 0x7fac301e3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:20.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac36ae7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac301419f0 0x7fac301e35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:20.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac36ae7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fac300b5b70 con 0x7fac300b4650 2024-03-16T10:11:20.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac36ae7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac300b59f0 con 0x7fac300b7650 2024-03-16T10:11:20.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac36ae7640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fac30154b30 con 0x7fac301419f0 2024-03-16T10:11:20.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac34883640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac300b4650 0x7fac30144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:20.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac34883640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac300b4650 0x7fac30144270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55228/0 (socket says 172.21.15.17:55228) 2024-03-16T10:11:20.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac34883640 1 -- 172.21.15.17:0/2663959675 learned_addr learned my addr 172.21.15.17:0/2663959675 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:20.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac34883640 1 -- 172.21.15.17:0/2663959675 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac301419f0 msgr2=0x7fac301e35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:20.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac34883640 1 --2- 172.21.15.17:0/2663959675 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac301419f0 0x7fac301e35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:20.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac34883640 1 -- 172.21.15.17:0/2663959675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac300b7650 msgr2=0x7fac301e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:20.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac34883640 1 --2- 172.21.15.17:0/2663959675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac300b7650 0x7fac301e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:20.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac34883640 1 -- 172.21.15.17:0/2663959675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac3014cca0 con 0x7fac300b4650 2024-03-16T10:11:20.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac34883640 1 --2- 172.21.15.17:0/2663959675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac300b4650 0x7fac30144270 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fac240190a0 tx=0x7fac24019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:20.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac2d7fa640 1 -- 172.21.15.17:0/2663959675 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac240b9070 con 0x7fac300b4650 2024-03-16T10:11:20.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac2d7fa640 1 -- 172.21.15.17:0/2663959675 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac240b4400 con 0x7fac300b4650 2024-03-16T10:11:20.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac36ae7640 1 -- 172.21.15.17:0/2663959675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac30147df0 con 0x7fac300b4650 2024-03-16T10:11:20.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac2d7fa640 1 -- 172.21.15.17:0/2663959675 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac240b0040 con 0x7fac300b4650 2024-03-16T10:11:20.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac36ae7640 1 -- 172.21.15.17:0/2663959675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac301447b0 con 0x7fac300b4650 2024-03-16T10:11:20.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac36ae7640 1 -- 172.21.15.17:0/2663959675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac301e2cd0 con 0x7fac300b4650 2024-03-16T10:11:20.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac2d7fa640 1 -- 172.21.15.17:0/2663959675 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fac240bc070 con 0x7fac300b4650 2024-03-16T10:11:20.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac2d7fa640 1 --2- 172.21.15.17:0/2663959675 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fac1c067660 0x7fac1c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:20.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac2d7fa640 1 -- 172.21.15.17:0/2663959675 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fac2411ee70 con 0x7fac300b4650 2024-03-16T10:11:20.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.032+0000 7fac2ffff640 1 --2- 172.21.15.17:0/2663959675 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fac1c067660 0x7fac1c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:20.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.036+0000 7fac2ffff640 1 --2- 172.21.15.17:0/2663959675 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fac1c067660 0x7fac1c069b20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fac200011f0 tx=0x7fac20007930 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:20.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.036+0000 7fac2d7fa640 1 -- 172.21.15.17:0/2663959675 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fac24016110 con 0x7fac300b4650 2024-03-16T10:11:20.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.188+0000 7fac36ae7640 1 -- 172.21.15.17:0/2663959675 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fac30068360 con 0x7fac300b4650 2024-03-16T10:11:20.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.192+0000 7fac2d7fa640 1 -- 172.21.15.17:0/2663959675 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fac240efb60 con 0x7fac300b4650 2024-03-16T10:11:20.195 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-16T10:11:20.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.192+0000 7fac36ae7640 1 -- 172.21.15.17:0/2663959675 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fac1c067660 msgr2=0x7fac1c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:20.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.192+0000 7fac36ae7640 1 --2- 172.21.15.17:0/2663959675 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fac1c067660 0x7fac1c069b20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fac200011f0 tx=0x7fac20007930 comp rx=0 tx=0).stop 2024-03-16T10:11:20.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.192+0000 7fac36ae7640 1 -- 172.21.15.17:0/2663959675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac300b4650 msgr2=0x7fac30144270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:20.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.192+0000 7fac36ae7640 1 --2- 172.21.15.17:0/2663959675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac300b4650 0x7fac30144270 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fac240190a0 tx=0x7fac24019230 comp rx=0 tx=0).stop 2024-03-16T10:11:20.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.192+0000 7fac36ae7640 1 -- 172.21.15.17:0/2663959675 shutdown_connections 2024-03-16T10:11:20.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.192+0000 7fac36ae7640 1 --2- 172.21.15.17:0/2663959675 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fac1c067660 0x7fac1c069b20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:20.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.192+0000 7fac36ae7640 1 --2- 172.21.15.17:0/2663959675 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fac301419f0 0x7fac301e35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:20.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.192+0000 7fac36ae7640 1 --2- 172.21.15.17:0/2663959675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac300b7650 0x7fac301e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:20.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.192+0000 7fac36ae7640 1 --2- 172.21.15.17:0/2663959675 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fac300b4650 0x7fac30144270 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:20.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.192+0000 7fac36ae7640 1 -- 172.21.15.17:0/2663959675 >> 172.21.15.17:0/2663959675 conn(0x7fac300bccd0 msgr2=0x7fac300bb330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:20.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.192+0000 7fac36ae7640 1 -- 172.21.15.17:0/2663959675 shutdown_connections 2024-03-16T10:11:20.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.192+0000 7fac36ae7640 1 -- 172.21.15.17:0/2663959675 wait complete. 2024-03-16T10:11:20.213 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2024-03-16T10:11:20.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.320+0000 7f84783c4640 1 Processor -- start 2024-03-16T10:11:20.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.320+0000 7f84783c4640 1 -- start start 2024-03-16T10:11:20.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.324+0000 7f84783c4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f847013e2f0 0x7f847014fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:20.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.324+0000 7f84783c4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84700b0200 con 0x7f84700673a0 2024-03-16T10:11:20.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.324+0000 7f84783c4640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8470070720 con 0x7f8470066fc0 2024-03-16T10:11:20.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.324+0000 7f84783c4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f84700b04e0 con 0x7f847013e2f0 2024-03-16T10:11:20.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.324+0000 7f8476961640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f847013e2f0 0x7f847014fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:20.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.324+0000 7f847595f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f84700673a0 0x7f84700b54b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46124/0 (socket says 172.21.15.17:46124) 2024-03-16T10:11:20.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.324+0000 7f847595f640 1 -- 172.21.15.17:0/89474620 learned_addr learned my addr 172.21.15.17:0/89474620 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:20.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.324+0000 7f8476961640 1 -- 172.21.15.17:0/89474620 >> v1:172.21.15.174:6789/0 conn(0x7f8470066fc0 legacy=0x7f84700af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:20.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.324+0000 7f8476961640 1 -- 172.21.15.17:0/89474620 >> v1:172.21.15.17:6789/0 conn(0x7f84700673a0 legacy=0x7f84700b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:20.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.324+0000 7f8476961640 1 -- 172.21.15.17:0/89474620 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84700ae330 con 0x7f847013e2f0 2024-03-16T10:11:20.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8476961640 1 --2- 172.21.15.17:0/89474620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f847013e2f0 0x7f847014fb40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8468009d90 tx=0x7f846801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd97271526e9b4f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:20.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f847515e640 1 -- 172.21.15.17:0/89474620 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84680b2070 con 0x7f847013e2f0 2024-03-16T10:11:20.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f847515e640 1 -- 172.21.15.17:0/89474620 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84680a5760 con 0x7f847013e2f0 2024-03-16T10:11:20.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f847515e640 1 -- 172.21.15.17:0/89474620 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84680a5b30 con 0x7f847013e2f0 2024-03-16T10:11:20.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 -- 172.21.15.17:0/89474620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f847013e2f0 msgr2=0x7f847014fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:20.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 --2- 172.21.15.17:0/89474620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f847013e2f0 0x7f847014fb40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8468009d90 tx=0x7f846801c8a0 comp rx=0 tx=0).stop 2024-03-16T10:11:20.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 -- 172.21.15.17:0/89474620 shutdown_connections 2024-03-16T10:11:20.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 --2- 172.21.15.17:0/89474620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f847013e2f0 0x7f847014fb40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:20.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 -- 172.21.15.17:0/89474620 >> 172.21.15.17:0/89474620 conn(0x7f84700b5e10 msgr2=0x7f84700b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:20.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 -- 172.21.15.17:0/89474620 shutdown_connections 2024-03-16T10:11:20.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 -- 172.21.15.17:0/89474620 wait complete. 2024-03-16T10:11:20.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 Processor -- start 2024-03-16T10:11:20.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 -- start start 2024-03-16T10:11:20.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8470066fc0 0x7f847013ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84700673a0 0x7f847013d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f847013e2f0 0x7f84701314d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f84700b5020 con 0x7f847013e2f0 2024-03-16T10:11:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84700b5bc0 con 0x7f8470066fc0 2024-03-16T10:11:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8470152ab0 con 0x7f84700673a0 2024-03-16T10:11:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8476961640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f847013e2f0 0x7f84701314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8476961640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f847013e2f0 0x7f84701314d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55236/0 (socket says 172.21.15.17:55236) 2024-03-16T10:11:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8476961640 1 -- 172.21.15.17:0/2685447589 learned_addr learned my addr 172.21.15.17:0/2685447589 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f847595f640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84700673a0 0x7f847013d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8476160640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8470066fc0 0x7f847013ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8476961640 1 -- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84700673a0 msgr2=0x7f847013d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8476961640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84700673a0 0x7f847013d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8476961640 1 -- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8470066fc0 msgr2=0x7f847013ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:20.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8476961640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8470066fc0 0x7f847013ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8476961640 1 -- 172.21.15.17:0/2685447589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f847013f320 con 0x7f847013e2f0 2024-03-16T10:11:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f847595f640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84700673a0 0x7f847013d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8476160640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8470066fc0 0x7f847013ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8476961640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f847013e2f0 0x7f84701314d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8468009d90 tx=0x7f8468019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8466ffd640 1 -- 172.21.15.17:0/2685447589 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84680bd020 con 0x7f847013e2f0 2024-03-16T10:11:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8466ffd640 1 -- 172.21.15.17:0/2685447589 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84680ad870 con 0x7f847013e2f0 2024-03-16T10:11:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 -- 172.21.15.17:0/2685447589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8470137e10 con 0x7f847013e2f0 2024-03-16T10:11:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f8466ffd640 1 -- 172.21.15.17:0/2685447589 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84680b2040 con 0x7f847013e2f0 2024-03-16T10:11:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.328+0000 7f84783c4640 1 -- 172.21.15.17:0/2685447589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f847013df40 con 0x7f847013e2f0 2024-03-16T10:11:20.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.332+0000 7f84783c4640 1 -- 172.21.15.17:0/2685447589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f847006a130 con 0x7f847013e2f0 2024-03-16T10:11:20.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.332+0000 7f8466ffd640 1 -- 172.21.15.17:0/2685447589 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f84680c0020 con 0x7f847013e2f0 2024-03-16T10:11:20.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.332+0000 7f8466ffd640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8454067690 0x7f8454069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:20.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.332+0000 7f8466ffd640 1 -- 172.21.15.17:0/2685447589 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8468120880 con 0x7f847013e2f0 2024-03-16T10:11:20.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.332+0000 7f8476160640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8454067690 0x7f8454069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:20.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.332+0000 7f8476160640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8454067690 0x7f8454069b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f846c00a850 tx=0x7f846c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:20.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.336+0000 7f8466ffd640 1 -- 172.21.15.17:0/2685447589 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f84680c7070 con 0x7f847013e2f0 2024-03-16T10:11:20.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:20.548+0000 7f84783c4640 1 -- 172.21.15.17:0/2685447589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) -- 0x7f8470142b00 con 0x7f847013e2f0 2024-03-16T10:11:21.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.248+0000 7f8466ffd640 1 -- 172.21.15.17:0/2685447589 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v46) ==== 137+0+0 (secure 0 0 0) 0x7f8468016520 con 0x7f847013e2f0 2024-03-16T10:11:21.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.252+0000 7f84783c4640 1 -- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8454067690 msgr2=0x7f8454069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:21.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.252+0000 7f84783c4640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8454067690 0x7f8454069b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f846c00a850 tx=0x7f846c008040 comp rx=0 tx=0).stop 2024-03-16T10:11:21.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.252+0000 7f84783c4640 1 -- 172.21.15.17:0/2685447589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f847013e2f0 msgr2=0x7f84701314d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:21.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.252+0000 7f84783c4640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f847013e2f0 0x7f84701314d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8468009d90 tx=0x7f8468019640 comp rx=0 tx=0).stop 2024-03-16T10:11:21.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.252+0000 7f84783c4640 1 -- 172.21.15.17:0/2685447589 shutdown_connections 2024-03-16T10:11:21.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.252+0000 7f84783c4640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8454067690 0x7f8454069b50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.252+0000 7f84783c4640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f847013e2f0 0x7f84701314d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.252+0000 7f84783c4640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f84700673a0 0x7f847013d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.252+0000 7f84783c4640 1 --2- 172.21.15.17:0/2685447589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8470066fc0 0x7f847013ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.252+0000 7f84783c4640 1 -- 172.21.15.17:0/2685447589 >> 172.21.15.17:0/2685447589 conn(0x7f84700b5e10 msgr2=0x7f84700b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:21.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.252+0000 7f84783c4640 1 -- 172.21.15.17:0/2685447589 shutdown_connections 2024-03-16T10:11:21.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.252+0000 7f84783c4640 1 -- 172.21.15.17:0/2685447589 wait complete. 2024-03-16T10:11:21.280 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-16T10:11:21.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f117e47b640 1 Processor -- start 2024-03-16T10:11:21.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f117e47b640 1 -- start start 2024-03-16T10:11:21.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f117e47b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f11780b4650 0x7f117814b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:21.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f117e47b640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f11780b7650 0x7f11781412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:21.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f117e47b640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1178141c30 con 0x7f1178141820 2024-03-16T10:11:21.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f117e47b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f11780b4a30 con 0x7f11780b4650 2024-03-16T10:11:21.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f117e47b640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f117814abe0 con 0x7f11780b7650 2024-03-16T10:11:21.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f1177fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f11780b4650 0x7f117814b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:21.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f1177fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f11780b4650 0x7f117814b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55246/0 (socket says 172.21.15.17:55246) 2024-03-16T10:11:21.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f1177fff640 1 -- 172.21.15.17:0/3271061500 learned_addr learned my addr 172.21.15.17:0/3271061500 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:21.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f11777fe640 1 --2- 172.21.15.17:0/3271061500 >> v2:172.21.15.174:3300/0 conn(0x7f11780b7650 0x7f11781412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:21.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f1177fff640 1 -- 172.21.15.17:0/3271061500 >> v2:172.21.15.174:3300/0 conn(0x7f11780b7650 msgr2=0x7f11781412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:21.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f1177fff640 1 --2- 172.21.15.17:0/3271061500 >> v2:172.21.15.174:3300/0 conn(0x7f11780b7650 0x7f11781412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f1177fff640 1 -- 172.21.15.17:0/3271061500 >> v1:172.21.15.174:6789/0 conn(0x7f1178141820 legacy=0x7f11781474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:21.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f1177fff640 1 -- 172.21.15.17:0/3271061500 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f117814aec0 con 0x7f11780b4650 2024-03-16T10:11:21.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f11777fe640 1 --2- 172.21.15.17:0/3271061500 >> v2:172.21.15.174:3300/0 conn(0x7f11780b7650 0x7f11781412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:21.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f1177fff640 1 --2- 172.21.15.17:0/3271061500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11780b4650 0x7f117814b3e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f11780bdd80 tx=0x7f116801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abf0a7a1d962469 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:21.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f1176ffd640 1 -- 172.21.15.17:0/3271061500 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680a5690 con 0x7f11780b4650 2024-03-16T10:11:21.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f1176ffd640 1 -- 172.21.15.17:0/3271061500 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11680a5830 con 0x7f11780b4650 2024-03-16T10:11:21.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.404+0000 7f1176ffd640 1 -- 172.21.15.17:0/3271061500 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11680a5c00 con 0x7f11780b4650 2024-03-16T10:11:21.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 -- 172.21.15.17:0/3271061500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11780b4650 msgr2=0x7f117814b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:21.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 --2- 172.21.15.17:0/3271061500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11780b4650 0x7f117814b3e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f11780bdd80 tx=0x7f116801c920 comp rx=0 tx=0).stop 2024-03-16T10:11:21.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 -- 172.21.15.17:0/3271061500 shutdown_connections 2024-03-16T10:11:21.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 --2- 172.21.15.17:0/3271061500 >> v2:172.21.15.174:3300/0 conn(0x7f11780b7650 0x7f11781412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 --2- 172.21.15.17:0/3271061500 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11780b4650 0x7f117814b3e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 -- 172.21.15.17:0/3271061500 >> 172.21.15.17:0/3271061500 conn(0x7f11780bccd0 msgr2=0x7f11780bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:21.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 -- 172.21.15.17:0/3271061500 shutdown_connections 2024-03-16T10:11:21.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 -- 172.21.15.17:0/3271061500 wait complete. 2024-03-16T10:11:21.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 Processor -- start 2024-03-16T10:11:21.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 -- start start 2024-03-16T10:11:21.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11780b4650 0x7f11781466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:21.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11780b7650 0x7f1178143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:21.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1178141820 0x7f11781e04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:21.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f11780b94a0 con 0x7f11780b7650 2024-03-16T10:11:21.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f11780b9320 con 0x7f11780b4650 2024-03-16T10:11:21.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117e47b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f11780b59f0 con 0x7f1178141820 2024-03-16T10:11:21.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f11777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11780b7650 0x7f1178143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:21.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f11777fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11780b7650 0x7f1178143ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55256/0 (socket says 172.21.15.17:55256) 2024-03-16T10:11:21.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f11777fe640 1 -- 172.21.15.17:0/2921549966 learned_addr learned my addr 172.21.15.17:0/2921549966 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:21.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f1177fff640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11780b4650 0x7f11781466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:21.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.408+0000 7f117ca18640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1178141820 0x7f11781e04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:21.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f11777fe640 1 -- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1178141820 msgr2=0x7f11781e04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:21.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f11777fe640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1178141820 0x7f11781e04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f11777fe640 1 -- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11780b4650 msgr2=0x7f11781466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:21.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f11777fe640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11780b4650 0x7f11781466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f11777fe640 1 -- 172.21.15.17:0/2921549966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11781440e0 con 0x7f11780b7650 2024-03-16T10:11:21.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f1177fff640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11780b4650 0x7f11781466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:21.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f11777fe640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11780b7650 0x7f1178143ba0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f116000c7c0 tx=0x7f116000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:21.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f1174ff9640 1 -- 172.21.15.17:0/2921549966 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1160007a30 con 0x7f11780b7650 2024-03-16T10:11:21.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f1174ff9640 1 -- 172.21.15.17:0/2921549966 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1160007bd0 con 0x7f11780b7650 2024-03-16T10:11:21.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f117e47b640 1 -- 172.21.15.17:0/2921549966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f117813ef30 con 0x7f11780b7650 2024-03-16T10:11:21.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f1174ff9640 1 -- 172.21.15.17:0/2921549966 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11600113f0 con 0x7f11780b7650 2024-03-16T10:11:21.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f117e47b640 1 -- 172.21.15.17:0/2921549966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1178144e20 con 0x7f11780b7650 2024-03-16T10:11:21.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f117e47b640 1 -- 172.21.15.17:0/2921549966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1178068360 con 0x7f11780b7650 2024-03-16T10:11:21.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f1174ff9640 1 -- 172.21.15.17:0/2921549966 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f1160011590 con 0x7f11780b7650 2024-03-16T10:11:21.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f1174ff9640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1158067660 0x7f1158069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:21.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f1174ff9640 1 -- 172.21.15.17:0/2921549966 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1160085a10 con 0x7f11780b7650 2024-03-16T10:11:21.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f1177fff640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1158067660 0x7f1158069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:21.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.412+0000 7f1177fff640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1158067660 0x7f1158069b20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11680ae000 tx=0x7f11680a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:21.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.416+0000 7f1174ff9640 1 -- 172.21.15.17:0/2921549966 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f11781440e0 con 0x7f11780b7650 2024-03-16T10:11:21.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.576+0000 7f117e47b640 1 -- 172.21.15.17:0/2921549966 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f11780b90c0 con 0x7f11780b7650 2024-03-16T10:11:21.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.580+0000 7f1174ff9640 1 -- 172.21.15.17:0/2921549966 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f1160015020 con 0x7f11780b7650 2024-03-16T10:11:21.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.580+0000 7f117e47b640 1 -- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1158067660 msgr2=0x7f1158069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:21.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.580+0000 7f117e47b640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1158067660 0x7f1158069b20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11680ae000 tx=0x7f11680a7070 comp rx=0 tx=0).stop 2024-03-16T10:11:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.580+0000 7f117e47b640 1 -- 172.21.15.17:0/2921549966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11780b7650 msgr2=0x7f1178143ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.580+0000 7f117e47b640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11780b7650 0x7f1178143ba0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f116000c7c0 tx=0x7f116000cc80 comp rx=0 tx=0).stop 2024-03-16T10:11:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.580+0000 7f117e47b640 1 -- 172.21.15.17:0/2921549966 shutdown_connections 2024-03-16T10:11:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.580+0000 7f117e47b640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1158067660 0x7f1158069b20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.580+0000 7f117e47b640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1178141820 0x7f11781e04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.580+0000 7f117e47b640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11780b7650 0x7f1178143ba0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.580+0000 7f117e47b640 1 --2- 172.21.15.17:0/2921549966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11780b4650 0x7f11781466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.580+0000 7f117e47b640 1 -- 172.21.15.17:0/2921549966 >> 172.21.15.17:0/2921549966 conn(0x7f11780bccd0 msgr2=0x7f11780bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.580+0000 7f117e47b640 1 -- 172.21.15.17:0/2921549966 shutdown_connections 2024-03-16T10:11:21.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.580+0000 7f117e47b640 1 -- 172.21.15.17:0/2921549966 wait complete. 2024-03-16T10:11:21.603 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-16T10:11:21.604 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-16T10:11:21.633 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-16T10:11:21.689 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-16T10:11:21.689 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:11:21.689 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T10:11:21.695 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:11:21.695 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T10:11:21.745 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:21.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.896+0000 7feb87fc8640 1 Processor -- start 2024-03-16T10:11:21.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.896+0000 7feb87fc8640 1 -- start start 2024-03-16T10:11:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.900+0000 7feb87fc8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feb80072c60 0x7feb801487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.900+0000 7feb87fc8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb80073040 con 0x7feb800b48f0 2024-03-16T10:11:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.900+0000 7feb87fc8640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb8013f990 con 0x7feb800b4da0 2024-03-16T10:11:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.900+0000 7feb87fc8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7feb80148e50 con 0x7feb80072c60 2024-03-16T10:11:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.900+0000 7feb85d64640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feb80072c60 0x7feb801487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.900+0000 7feb85563640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7feb800b48f0 0x7feb8013f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46126/0 (socket says 172.21.15.17:46126) 2024-03-16T10:11:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.900+0000 7feb85563640 1 -- 172.21.15.17:0/2625548218 learned_addr learned my addr 172.21.15.17:0/2625548218 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:21.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.900+0000 7feb84d62640 1 -- 172.21.15.17:0/2625548218 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 498759088 0 0) 0x7feb80073040 con 0x7feb800b48f0 2024-03-16T10:11:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.900+0000 7feb84d62640 1 -- 172.21.15.17:0/2625548218 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb6c003530 con 0x7feb800b48f0 2024-03-16T10:11:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.900+0000 7feb85d64640 1 -- 172.21.15.17:0/2625548218 >> v1:172.21.15.174:6789/0 conn(0x7feb800b4da0 legacy=0x7feb800b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.900+0000 7feb85d64640 1 -- 172.21.15.17:0/2625548218 >> v1:172.21.15.17:6789/0 conn(0x7feb800b48f0 legacy=0x7feb8013f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:21.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.900+0000 7feb85d64640 1 -- 172.21.15.17:0/2625548218 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb80149130 con 0x7feb80072c60 2024-03-16T10:11:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.904+0000 7feb85d64640 1 --2- 172.21.15.17:0/2625548218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80072c60 0x7feb801487c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feb78009d90 tx=0x7feb7801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc307cdb35eb382d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.904+0000 7feb84d62640 1 -- 172.21.15.17:0/2625548218 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb780b2070 con 0x7feb80072c60 2024-03-16T10:11:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.904+0000 7feb84d62640 1 -- 172.21.15.17:0/2625548218 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb780a5710 con 0x7feb80072c60 2024-03-16T10:11:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.904+0000 7feb84d62640 1 -- 172.21.15.17:0/2625548218 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb780a5ac0 con 0x7feb80072c60 2024-03-16T10:11:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.904+0000 7feb87fc8640 1 -- 172.21.15.17:0/2625548218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80072c60 msgr2=0x7feb801487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.904+0000 7feb87fc8640 1 --2- 172.21.15.17:0/2625548218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80072c60 0x7feb801487c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feb78009d90 tx=0x7feb7801c850 comp rx=0 tx=0).stop 2024-03-16T10:11:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.904+0000 7feb87fc8640 1 -- 172.21.15.17:0/2625548218 shutdown_connections 2024-03-16T10:11:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.904+0000 7feb87fc8640 1 --2- 172.21.15.17:0/2625548218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80072c60 0x7feb801487c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.904+0000 7feb87fc8640 1 -- 172.21.15.17:0/2625548218 >> 172.21.15.17:0/2625548218 conn(0x7feb800b9a30 msgr2=0x7feb800b9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.904+0000 7feb87fc8640 1 -- 172.21.15.17:0/2625548218 shutdown_connections 2024-03-16T10:11:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.904+0000 7feb87fc8640 1 -- 172.21.15.17:0/2625548218 wait complete. 2024-03-16T10:11:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.904+0000 7feb87fc8640 1 Processor -- start 2024-03-16T10:11:21.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.904+0000 7feb87fc8640 1 -- start start 2024-03-16T10:11:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb87fc8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80072c60 0x7feb801e1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb87fc8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb800b48f0 0x7feb80140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb87fc8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb800b4da0 0x7feb801406a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb87fc8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7feb80153ae0 con 0x7feb80072c60 2024-03-16T10:11:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb87fc8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb80153960 con 0x7feb800b48f0 2024-03-16T10:11:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb87fc8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7feb80153c60 con 0x7feb800b4da0 2024-03-16T10:11:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb85d64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80072c60 0x7feb801e1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb85d64640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80072c60 0x7feb801e1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55274/0 (socket says 172.21.15.17:55274) 2024-03-16T10:11:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb85d64640 1 -- 172.21.15.17:0/143737891 learned_addr learned my addr 172.21.15.17:0/143737891 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb85563640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb800b48f0 0x7feb80140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb85d64640 1 -- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb800b4da0 msgr2=0x7feb801406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb86565640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb800b4da0 0x7feb801406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb85d64640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb800b4da0 0x7feb801406a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb85d64640 1 -- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb800b48f0 msgr2=0x7feb80140160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb85d64640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb800b48f0 0x7feb80140160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb85d64640 1 -- 172.21.15.17:0/143737891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb800afc80 con 0x7feb80072c60 2024-03-16T10:11:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb86565640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb800b4da0 0x7feb801406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb85563640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb800b48f0 0x7feb80140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb85d64640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80072c60 0x7feb801e1150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb780b0040 tx=0x7feb780195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb727fc640 1 -- 172.21.15.17:0/143737891 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb780bd020 con 0x7feb80072c60 2024-03-16T10:11:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb727fc640 1 -- 172.21.15.17:0/143737891 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb780a5e50 con 0x7feb80072c60 2024-03-16T10:11:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb87fc8640 1 -- 172.21.15.17:0/143737891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb8013df30 con 0x7feb80072c60 2024-03-16T10:11:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb727fc640 1 -- 172.21.15.17:0/143737891 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb780b2040 con 0x7feb80072c60 2024-03-16T10:11:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb87fc8640 1 -- 172.21.15.17:0/143737891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb801e1c30 con 0x7feb80072c60 2024-03-16T10:11:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb87fc8640 1 -- 172.21.15.17:0/143737891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb8006a130 con 0x7feb80072c60 2024-03-16T10:11:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb727fc640 1 -- 172.21.15.17:0/143737891 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7feb780c0020 con 0x7feb80072c60 2024-03-16T10:11:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb727fc640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feb64067660 0x7feb64069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb727fc640 1 -- 172.21.15.17:0/143737891 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feb78120880 con 0x7feb80072c60 2024-03-16T10:11:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.908+0000 7feb85563640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feb64067660 0x7feb64069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.912+0000 7feb85563640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feb64067660 0x7feb64069b20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feb8013e720 tx=0x7feb74009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:21.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:21.916+0000 7feb727fc640 1 -- 172.21.15.17:0/143737891 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7feb780c7030 con 0x7feb80072c60 2024-03-16T10:11:22.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.072+0000 7feb87fc8640 1 -- 172.21.15.17:0/143737891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb800b4650 con 0x7feb80072c60 2024-03-16T10:11:22.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.076+0000 7feb727fc640 1 -- 172.21.15.17:0/143737891 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7feb78016560 con 0x7feb80072c60 2024-03-16T10:11:22.081 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:22.081 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:22.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.076+0000 7feb87fc8640 1 -- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feb64067660 msgr2=0x7feb64069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.076+0000 7feb87fc8640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feb64067660 0x7feb64069b20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feb8013e720 tx=0x7feb74009040 comp rx=0 tx=0).stop 2024-03-16T10:11:22.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.076+0000 7feb87fc8640 1 -- 172.21.15.17:0/143737891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80072c60 msgr2=0x7feb801e1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.076+0000 7feb87fc8640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80072c60 0x7feb801e1150 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb780b0040 tx=0x7feb780195f0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.076+0000 7feb87fc8640 1 -- 172.21.15.17:0/143737891 shutdown_connections 2024-03-16T10:11:22.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.076+0000 7feb87fc8640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feb64067660 0x7feb64069b20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.076+0000 7feb87fc8640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb800b4da0 0x7feb801406a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.076+0000 7feb87fc8640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb800b48f0 0x7feb80140160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.076+0000 7feb87fc8640 1 --2- 172.21.15.17:0/143737891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb80072c60 0x7feb801e1150 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.076+0000 7feb87fc8640 1 -- 172.21.15.17:0/143737891 >> 172.21.15.17:0/143737891 conn(0x7feb800b9a30 msgr2=0x7feb80144b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.076+0000 7feb87fc8640 1 -- 172.21.15.17:0/143737891 shutdown_connections 2024-03-16T10:11:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.076+0000 7feb87fc8640 1 -- 172.21.15.17:0/143737891 wait complete. 2024-03-16T10:11:22.098 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:10:55.723734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:10:56.459894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:11:22.099 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:11:22.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8ab5933640 1 Processor -- start 2024-03-16T10:11:22.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8ab5933640 1 -- start start 2024-03-16T10:11:22.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8ab5933640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8ab0067220 0x7f8ab00b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8ab5933640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ab00b3da0 0x7f8ab0147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8ab5933640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ab00b05a0 con 0x7f8ab00b4200 2024-03-16T10:11:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8ab5933640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8ab006e4d0 con 0x7f8ab0067220 2024-03-16T10:11:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8ab5933640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab0067600 con 0x7f8ab00b3da0 2024-03-16T10:11:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8aaf7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8ab00b4200 0x7f8ab014ad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46132/0 (socket says 172.21.15.17:46132) 2024-03-16T10:11:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8aaf7fe640 1 -- 172.21.15.17:0/3985433258 learned_addr learned my addr 172.21.15.17:0/3985433258 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8aaeffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8ab0067220 0x7f8ab00b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8aae7fc640 1 --2- 172.21.15.17:0/3985433258 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ab00b3da0 0x7f8ab0147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8aaeffd640 1 -- 172.21.15.17:0/3985433258 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ab00b3da0 msgr2=0x7f8ab0147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8aaeffd640 1 --2- 172.21.15.17:0/3985433258 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ab00b3da0 0x7f8ab0147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8aaeffd640 1 -- 172.21.15.17:0/3985433258 >> v1:172.21.15.17:6789/0 conn(0x7f8ab00b4200 legacy=0x7f8ab014ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8aaeffd640 1 -- 172.21.15.17:0/3985433258 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab006e1f0 con 0x7f8ab0067220 2024-03-16T10:11:22.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.232+0000 7f8aadffb640 1 -- 172.21.15.17:0/3985433258 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1980392863 0 0) 0x7f8ab00b05a0 con 0x7f8ab00b4200 2024-03-16T10:11:22.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aaeffd640 1 --2- 172.21.15.17:0/3985433258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab0067220 0x7f8ab00b0060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40097c0 tx=0x7f8aa401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64dfeb553e283549 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:22.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aadffb640 1 -- 172.21.15.17:0/3985433258 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40b0070 con 0x7f8ab0067220 2024-03-16T10:11:22.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aadffb640 1 -- 172.21.15.17:0/3985433258 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa40ab600 con 0x7f8ab0067220 2024-03-16T10:11:22.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aadffb640 1 -- 172.21.15.17:0/3985433258 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40ab9d0 con 0x7f8ab0067220 2024-03-16T10:11:22.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 -- 172.21.15.17:0/3985433258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab0067220 msgr2=0x7f8ab00b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 --2- 172.21.15.17:0/3985433258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab0067220 0x7f8ab00b0060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40097c0 tx=0x7f8aa401c4a0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 -- 172.21.15.17:0/3985433258 shutdown_connections 2024-03-16T10:11:22.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 --2- 172.21.15.17:0/3985433258 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ab00b3da0 0x7f8ab0147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 --2- 172.21.15.17:0/3985433258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab0067220 0x7f8ab00b0060 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 -- 172.21.15.17:0/3985433258 >> 172.21.15.17:0/3985433258 conn(0x7f8ab00b5ea0 msgr2=0x7f8ab00b6280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:22.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 -- 172.21.15.17:0/3985433258 shutdown_connections 2024-03-16T10:11:22.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 -- 172.21.15.17:0/3985433258 wait complete. 2024-03-16T10:11:22.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 Processor -- start 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 -- start start 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ab0067220 0x7f8ab0147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ab00b3da0 0x7f8ab01e3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab00b4200 0x7f8ab013f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab006df90 con 0x7f8ab00b4200 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab006de10 con 0x7f8ab0067220 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8ab5933640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab0155060 con 0x7f8ab00b3da0 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aaf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab00b4200 0x7f8ab013f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aaf7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab00b4200 0x7f8ab013f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55284/0 (socket says 172.21.15.17:55284) 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aaf7fe640 1 -- 172.21.15.17:0/1058930992 learned_addr learned my addr 172.21.15.17:0/1058930992 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aae7fc640 1 --2- 172.21.15.17:0/1058930992 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ab00b3da0 0x7f8ab01e3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aaeffd640 1 --2- 172.21.15.17:0/1058930992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ab0067220 0x7f8ab0147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aaf7fe640 1 -- 172.21.15.17:0/1058930992 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ab00b3da0 msgr2=0x7f8ab01e3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aaf7fe640 1 --2- 172.21.15.17:0/1058930992 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ab00b3da0 0x7f8ab01e3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aaf7fe640 1 -- 172.21.15.17:0/1058930992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ab0067220 msgr2=0x7f8ab0147610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aaf7fe640 1 --2- 172.21.15.17:0/1058930992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ab0067220 0x7f8ab0147610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aaf7fe640 1 -- 172.21.15.17:0/1058930992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab0152e00 con 0x7f8ab00b4200 2024-03-16T10:11:22.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8aaf7fe640 1 --2- 172.21.15.17:0/1058930992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab00b4200 0x7f8ab013f340 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800c7a0 tx=0x7f8aa800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:22.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.236+0000 7f8a97fff640 1 -- 172.21.15.17:0/1058930992 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8018070 con 0x7f8ab00b4200 2024-03-16T10:11:22.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.240+0000 7f8a97fff640 1 -- 172.21.15.17:0/1058930992 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8aa800a5e0 con 0x7f8ab00b4200 2024-03-16T10:11:22.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.240+0000 7f8ab5933640 1 -- 172.21.15.17:0/1058930992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab01404c0 con 0x7f8ab00b4200 2024-03-16T10:11:22.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.240+0000 7f8a97fff640 1 -- 172.21.15.17:0/1058930992 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa800aa50 con 0x7f8ab00b4200 2024-03-16T10:11:22.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.240+0000 7f8ab5933640 1 -- 172.21.15.17:0/1058930992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab0149d80 con 0x7f8ab00b4200 2024-03-16T10:11:22.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.240+0000 7f8a95ffb640 1 -- 172.21.15.17:0/1058930992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a880052a0 con 0x7f8ab00b4200 2024-03-16T10:11:22.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.240+0000 7f8a97fff640 1 -- 172.21.15.17:0/1058930992 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8aa801b070 con 0x7f8ab00b4200 2024-03-16T10:11:22.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.240+0000 7f8a97fff640 1 --2- 172.21.15.17:0/1058930992 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8a9c067660 0x7f8a9c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.240+0000 7f8a97fff640 1 -- 172.21.15.17:0/1058930992 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8aa80851a0 con 0x7f8ab00b4200 2024-03-16T10:11:22.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.240+0000 7f8aaeffd640 1 --2- 172.21.15.17:0/1058930992 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8a9c067660 0x7f8a9c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.240+0000 7f8aaeffd640 1 --2- 172.21.15.17:0/1058930992 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8a9c067660 0x7f8a9c069b20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40ad000 tx=0x7f8aa40a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:22.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.244+0000 7f8a97fff640 1 -- 172.21.15.17:0/1058930992 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8aa8055f10 con 0x7f8ab00b4200 2024-03-16T10:11:22.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.404+0000 7f8a95ffb640 1 -- 172.21.15.17:0/1058930992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8a88005860 con 0x7f8ab00b4200 2024-03-16T10:11:22.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.408+0000 7f8a97fff640 1 -- 172.21.15.17:0/1058930992 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f8aa8055810 con 0x7f8ab00b4200 2024-03-16T10:11:22.412 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:11:22.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.408+0000 7f8a95ffb640 1 -- 172.21.15.17:0/1058930992 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8a9c067660 msgr2=0x7f8a9c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.408+0000 7f8a95ffb640 1 --2- 172.21.15.17:0/1058930992 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8a9c067660 0x7f8a9c069b20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8aa40ad000 tx=0x7f8aa40a6000 comp rx=0 tx=0).stop 2024-03-16T10:11:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.408+0000 7f8a95ffb640 1 -- 172.21.15.17:0/1058930992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab00b4200 msgr2=0x7f8ab013f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.408+0000 7f8a95ffb640 1 --2- 172.21.15.17:0/1058930992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab00b4200 0x7f8ab013f340 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800c7a0 tx=0x7f8aa800cc60 comp rx=0 tx=0).stop 2024-03-16T10:11:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.408+0000 7f8a95ffb640 1 -- 172.21.15.17:0/1058930992 shutdown_connections 2024-03-16T10:11:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.408+0000 7f8a95ffb640 1 --2- 172.21.15.17:0/1058930992 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8a9c067660 0x7f8a9c069b20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.408+0000 7f8a95ffb640 1 --2- 172.21.15.17:0/1058930992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab00b4200 0x7f8ab013f340 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.408+0000 7f8a95ffb640 1 --2- 172.21.15.17:0/1058930992 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8ab00b3da0 0x7f8ab01e3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.408+0000 7f8a95ffb640 1 --2- 172.21.15.17:0/1058930992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ab0067220 0x7f8ab0147610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.408+0000 7f8a95ffb640 1 -- 172.21.15.17:0/1058930992 >> 172.21.15.17:0/1058930992 conn(0x7f8ab00b5ea0 msgr2=0x7f8ab009ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.408+0000 7f8a95ffb640 1 -- 172.21.15.17:0/1058930992 shutdown_connections 2024-03-16T10:11:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.408+0000 7f8a95ffb640 1 -- 172.21.15.17:0/1058930992 wait complete. 2024-03-16T10:11:22.431 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:11:22.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f201ade2640 1 Processor -- start 2024-03-16T10:11:22.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f201ade2640 1 -- start start 2024-03-16T10:11:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f201ade2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20140b7770 0x7f2014145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f201ade2640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f20140b49d0 0x7f201414faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f201ade2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20140b6fd0 con 0x7f2014073070 2024-03-16T10:11:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f201ade2640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f20140b64f0 con 0x7f20140b49d0 2024-03-16T10:11:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f201ade2640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f20140b67d0 con 0x7f20140b7770 2024-03-16T10:11:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f2013fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2014073070 0x7f20140b42c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46140/0 (socket says 172.21.15.17:46140) 2024-03-16T10:11:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f2013fff640 1 -- 172.21.15.17:0/1294249270 learned_addr learned my addr 172.21.15.17:0/1294249270 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f2018b7e640 1 --2- 172.21.15.17:0/1294249270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20140b7770 0x7f2014145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f201937f640 1 --2- 172.21.15.17:0/1294249270 >> v2:172.21.15.174:3300/0 conn(0x7f20140b49d0 0x7f201414faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 -- 172.21.15.17:0/1294249270 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4214906311 0 0) 0x7f20140b6fd0 con 0x7f2014073070 2024-03-16T10:11:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 -- 172.21.15.17:0/1294249270 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ffc003530 con 0x7f2014073070 2024-03-16T10:11:22.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 -- 172.21.15.17:0/1294249270 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 682068706 0 0) 0x7f1ffc003530 con 0x7f2014073070 2024-03-16T10:11:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 -- 172.21.15.17:0/1294249270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20140b7770 msgr2=0x7f2014145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 --2- 172.21.15.17:0/1294249270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20140b7770 0x7f2014145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 -- 172.21.15.17:0/1294249270 >> v2:172.21.15.174:3300/0 conn(0x7f20140b49d0 msgr2=0x7f201414faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 --2- 172.21.15.17:0/1294249270 >> v2:172.21.15.174:3300/0 conn(0x7f20140b49d0 0x7f201414faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 -- 172.21.15.17:0/1294249270 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20140b6ab0 con 0x7f2014073070 2024-03-16T10:11:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 -- 172.21.15.17:0/1294249270 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f2008001680 con 0x7f2014073070 2024-03-16T10:11:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 -- 172.21.15.17:0/1294249270 >> v1:172.21.15.17:6789/0 conn(0x7f2014073070 legacy=0x7f20140b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 --2- 172.21.15.17:0/1294249270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20140b7770 0x7f1ffc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 --2- 172.21.15.17:0/1294249270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ffc003f40 0x7f1ffc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 --2- 172.21.15.17:0/1294249270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ffc004d50 0x7f1ffc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 -- 172.21.15.17:0/1294249270 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f20140b64f0 con 0x7f1ffc004d50 2024-03-16T10:11:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 -- 172.21.15.17:0/1294249270 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f20140b67d0 con 0x7f1ffc003f40 2024-03-16T10:11:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f20137fe640 1 -- 172.21.15.17:0/1294249270 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1ffc00bab0 con 0x7f20140b7770 2024-03-16T10:11:22.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f201937f640 1 --2- 172.21.15.17:0/1294249270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ffc004d50 0x7f1ffc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f2013fff640 1 --2- 172.21.15.17:0/1294249270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ffc003f40 0x7f1ffc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f2018b7e640 1 --2- 172.21.15.17:0/1294249270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20140b7770 0x7f1ffc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f201937f640 1 -- 172.21.15.17:0/1294249270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20140b7770 msgr2=0x7f1ffc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f201937f640 1 --2- 172.21.15.17:0/1294249270 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f20140b7770 0x7f1ffc0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f201937f640 1 -- 172.21.15.17:0/1294249270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ffc003f40 msgr2=0x7f1ffc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f201937f640 1 --2- 172.21.15.17:0/1294249270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ffc003f40 0x7f1ffc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.548+0000 7f201937f640 1 -- 172.21.15.17:0/1294249270 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ffc003530 con 0x7f1ffc004d50 2024-03-16T10:11:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201937f640 1 --2- 172.21.15.17:0/1294249270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ffc004d50 0x7f1ffc0051d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f200c009ae0 tx=0x7f200c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c38e2aa64a8637 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f20137fe640 1 -- 172.21.15.17:0/1294249270 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0b2070 con 0x7f1ffc004d50 2024-03-16T10:11:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f20137fe640 1 -- 172.21.15.17:0/1294249270 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f200c0a5590 con 0x7f1ffc004d50 2024-03-16T10:11:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f20137fe640 1 -- 172.21.15.17:0/1294249270 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c0096b0 con 0x7f1ffc004d50 2024-03-16T10:11:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201ade2640 1 -- 172.21.15.17:0/1294249270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ffc004d50 msgr2=0x7f1ffc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201ade2640 1 --2- 172.21.15.17:0/1294249270 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ffc004d50 0x7f1ffc0051d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f200c009ae0 tx=0x7f200c01cab0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201937f640 1 -- 172.21.15.17:0/1294249270 reap_dead start 2024-03-16T10:11:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201ade2640 1 -- 172.21.15.17:0/1294249270 shutdown_connections 2024-03-16T10:11:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201ade2640 1 -- 172.21.15.17:0/1294249270 >> 172.21.15.17:0/1294249270 conn(0x7f201406b290 msgr2=0x7f201406b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201ade2640 1 -- 172.21.15.17:0/1294249270 shutdown_connections 2024-03-16T10:11:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201ade2640 1 -- 172.21.15.17:0/1294249270 wait complete. 2024-03-16T10:11:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201ade2640 1 Processor -- start 2024-03-16T10:11:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201ade2640 1 -- start start 2024-03-16T10:11:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201ade2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2014142520 0x7f201413bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201ade2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201413c420 0x7f2014146640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201ade2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2014146b80 0x7f201406c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201ade2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2014150370 con 0x7f2014142520 2024-03-16T10:11:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201ade2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f20141501f0 con 0x7f201413c420 2024-03-16T10:11:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.552+0000 7f201ade2640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f20141504f0 con 0x7f2014146b80 2024-03-16T10:11:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f2018b7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2014142520 0x7f201413bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f2018b7e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2014142520 0x7f201413bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55294/0 (socket says 172.21.15.17:55294) 2024-03-16T10:11:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f2018b7e640 1 -- 172.21.15.17:0/3807368965 learned_addr learned my addr 172.21.15.17:0/3807368965 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f201937f640 1 --2- 172.21.15.17:0/3807368965 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2014146b80 0x7f201406c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f2018b7e640 1 -- 172.21.15.17:0/3807368965 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2014146b80 msgr2=0x7f201406c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f2018b7e640 1 --2- 172.21.15.17:0/3807368965 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2014146b80 0x7f201406c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f2018b7e640 1 -- 172.21.15.17:0/3807368965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201413c420 msgr2=0x7f2014146640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f2013fff640 1 --2- 172.21.15.17:0/3807368965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201413c420 0x7f2014146640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f2018b7e640 1 --2- 172.21.15.17:0/3807368965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201413c420 0x7f2014146640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f2018b7e640 1 -- 172.21.15.17:0/3807368965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2014148d20 con 0x7f2014142520 2024-03-16T10:11:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f2018b7e640 1 --2- 172.21.15.17:0/3807368965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2014142520 0x7f201413bee0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2004007f30 tx=0x7f2004011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:22.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f20117fa640 1 -- 172.21.15.17:0/3807368965 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200401a020 con 0x7f2014142520 2024-03-16T10:11:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f20117fa640 1 -- 172.21.15.17:0/3807368965 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2004016470 con 0x7f2014142520 2024-03-16T10:11:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f20117fa640 1 -- 172.21.15.17:0/3807368965 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004016860 con 0x7f2014142520 2024-03-16T10:11:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f201ade2640 1 -- 172.21.15.17:0/3807368965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2014147000 con 0x7f2014142520 2024-03-16T10:11:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f201ade2640 1 -- 172.21.15.17:0/3807368965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2014142900 con 0x7f2014142520 2024-03-16T10:11:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f201ade2640 1 -- 172.21.15.17:0/3807368965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f201406a130 con 0x7f2014142520 2024-03-16T10:11:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f20117fa640 1 -- 172.21.15.17:0/3807368965 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f200401d020 con 0x7f2014142520 2024-03-16T10:11:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f20117fa640 1 --2- 172.21.15.17:0/3807368965 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2000067660 0x7f2000069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f20117fa640 1 -- 172.21.15.17:0/3807368965 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f20040867c0 con 0x7f2014142520 2024-03-16T10:11:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.556+0000 7f2013fff640 1 --2- 172.21.15.17:0/3807368965 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2000067660 0x7f2000069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.560+0000 7f2013fff640 1 --2- 172.21.15.17:0/3807368965 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2000067660 0x7f2000069b20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2008000fc0 tx=0x7f20080047f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:22.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.560+0000 7f20117fa640 1 -- 172.21.15.17:0/3807368965 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2004024030 con 0x7f2014142520 2024-03-16T10:11:22.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.724+0000 7f201ade2640 1 -- 172.21.15.17:0/3807368965 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2014142c10 con 0x7f2014142520 2024-03-16T10:11:22.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.724+0000 7f20117fa640 1 -- 172.21.15.17:0/3807368965 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f2014147000 con 0x7f2014142520 2024-03-16T10:11:22.730 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-16T10:11:22.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.728+0000 7f201ade2640 1 -- 172.21.15.17:0/3807368965 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2000067660 msgr2=0x7f2000069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.728+0000 7f201ade2640 1 --2- 172.21.15.17:0/3807368965 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2000067660 0x7f2000069b20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2008000fc0 tx=0x7f20080047f0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.728+0000 7f201ade2640 1 -- 172.21.15.17:0/3807368965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2014142520 msgr2=0x7f201413bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.728+0000 7f201ade2640 1 --2- 172.21.15.17:0/3807368965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2014142520 0x7f201413bee0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2004007f30 tx=0x7f2004011660 comp rx=0 tx=0).stop 2024-03-16T10:11:22.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.728+0000 7f201ade2640 1 -- 172.21.15.17:0/3807368965 shutdown_connections 2024-03-16T10:11:22.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.728+0000 7f201ade2640 1 --2- 172.21.15.17:0/3807368965 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2000067660 0x7f2000069b20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.728+0000 7f201ade2640 1 --2- 172.21.15.17:0/3807368965 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2014146b80 0x7f201406c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.728+0000 7f201ade2640 1 --2- 172.21.15.17:0/3807368965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f201413c420 0x7f2014146640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.728+0000 7f201ade2640 1 --2- 172.21.15.17:0/3807368965 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2014142520 0x7f201413bee0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.728+0000 7f201ade2640 1 -- 172.21.15.17:0/3807368965 >> 172.21.15.17:0/3807368965 conn(0x7f201406b290 msgr2=0x7f2014143170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:22.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.728+0000 7f201ade2640 1 -- 172.21.15.17:0/3807368965 shutdown_connections 2024-03-16T10:11:22.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.728+0000 7f201ade2640 1 -- 172.21.15.17:0/3807368965 wait complete. 2024-03-16T10:11:22.749 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:11:22.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04b0c78640 1 Processor -- start 2024-03-16T10:11:22.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04b0c78640 1 -- start start 2024-03-16T10:11:22.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04b0c78640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f04ac06e3c0 0x7f04ac0af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04b0c78640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac148470 0x7f04ac151e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04b0c78640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04ac06e7a0 con 0x7f04ac0af8d0 2024-03-16T10:11:22.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04b0c78640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f04ac09f260 con 0x7f04ac06e3c0 2024-03-16T10:11:22.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04b0c78640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f04ac0aec20 con 0x7f04ac148470 2024-03-16T10:11:22.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04aa59c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f04ac06e3c0 0x7f04ac0af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04a9d9b640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f04ac0af8d0 0x7f04ac147d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.17:59162/0 (socket says 172.21.15.17:59162) 2024-03-16T10:11:22.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04a9d9b640 1 -- 172.21.15.17:0/4095163209 learned_addr learned my addr 172.21.15.17:0/4095163209 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:22.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04aad9d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac148470 0x7f04ac151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04aad9d640 1 -- 172.21.15.17:0/4095163209 >> v2:172.21.15.174:3300/0 conn(0x7f04ac06e3c0 msgr2=0x7f04ac0af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:22.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04aad9d640 1 --2- 172.21.15.17:0/4095163209 >> v2:172.21.15.174:3300/0 conn(0x7f04ac06e3c0 0x7f04ac0af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04aad9d640 1 -- 172.21.15.17:0/4095163209 >> v1:172.21.15.174:6789/0 conn(0x7f04ac0af8d0 legacy=0x7f04ac147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04aad9d640 1 -- 172.21.15.17:0/4095163209 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac06dd10 con 0x7f04ac148470 2024-03-16T10:11:22.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.876+0000 7f04aa59c640 1 --2- 172.21.15.17:0/4095163209 >> v2:172.21.15.174:3300/0 conn(0x7f04ac06e3c0 0x7f04ac0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:11:22.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.880+0000 7f04aad9d640 1 --2- 172.21.15.17:0/4095163209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac148470 0x7f04ac151e50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f049c0097c0 tx=0x7f049c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da14845209a7a5e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:22.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.880+0000 7f04a959a640 1 -- 172.21.15.17:0/4095163209 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c0b0070 con 0x7f04ac148470 2024-03-16T10:11:22.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.880+0000 7f04a959a640 1 -- 172.21.15.17:0/4095163209 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f049c0ab600 con 0x7f04ac148470 2024-03-16T10:11:22.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.880+0000 7f04a959a640 1 -- 172.21.15.17:0/4095163209 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c0ab9d0 con 0x7f04ac148470 2024-03-16T10:11:22.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.880+0000 7f04b0c78640 1 -- 172.21.15.17:0/4095163209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac148470 msgr2=0x7f04ac151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.880+0000 7f04b0c78640 1 --2- 172.21.15.17:0/4095163209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac148470 0x7f04ac151e50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f049c0097c0 tx=0x7f049c01c4a0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.880+0000 7f04b0c78640 1 -- 172.21.15.17:0/4095163209 shutdown_connections 2024-03-16T10:11:22.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.880+0000 7f04b0c78640 1 --2- 172.21.15.17:0/4095163209 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac148470 0x7f04ac151e50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.880+0000 7f04b0c78640 1 --2- 172.21.15.17:0/4095163209 >> v2:172.21.15.174:3300/0 conn(0x7f04ac06e3c0 0x7f04ac0af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.880+0000 7f04b0c78640 1 -- 172.21.15.17:0/4095163209 >> 172.21.15.17:0/4095163209 conn(0x7f04ac0b65d0 msgr2=0x7f04ac0b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:22.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.880+0000 7f04b0c78640 1 -- 172.21.15.17:0/4095163209 shutdown_connections 2024-03-16T10:11:22.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.880+0000 7f04b0c78640 1 -- 172.21.15.17:0/4095163209 wait complete. 2024-03-16T10:11:22.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.880+0000 7f04b0c78640 1 Processor -- start 2024-03-16T10:11:22.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.880+0000 7f04b0c78640 1 -- start start 2024-03-16T10:11:22.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04b0c78640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04ac06e3c0 0x7f04ac0b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04b0c78640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac0af8d0 0x7f04ac13eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04b0c78640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04ac148470 0x7f04ac1e01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04b0c78640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac06d4d0 con 0x7f04ac148470 2024-03-16T10:11:22.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04b0c78640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04ac06f070 con 0x7f04ac06e3c0 2024-03-16T10:11:22.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04b0c78640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f04ac154bb0 con 0x7f04ac0af8d0 2024-03-16T10:11:22.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04aad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04ac148470 0x7f04ac1e01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04aad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04ac148470 0x7f04ac1e01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55304/0 (socket says 172.21.15.17:55304) 2024-03-16T10:11:22.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04aad9d640 1 -- 172.21.15.17:0/3529628532 learned_addr learned my addr 172.21.15.17:0/3529628532 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:22.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04aa59c640 1 --2- 172.21.15.17:0/3529628532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04ac06e3c0 0x7f04ac0b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04a9d9b640 1 --2- 172.21.15.17:0/3529628532 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac0af8d0 0x7f04ac13eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04aad9d640 1 -- 172.21.15.17:0/3529628532 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac0af8d0 msgr2=0x7f04ac13eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04aad9d640 1 --2- 172.21.15.17:0/3529628532 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac0af8d0 0x7f04ac13eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04aad9d640 1 -- 172.21.15.17:0/3529628532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04ac06e3c0 msgr2=0x7f04ac0b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:22.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04aad9d640 1 --2- 172.21.15.17:0/3529628532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04ac06e3c0 0x7f04ac0b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:22.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04aad9d640 1 -- 172.21.15.17:0/3529628532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04ac0b3e50 con 0x7f04ac148470 2024-03-16T10:11:22.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04aad9d640 1 --2- 172.21.15.17:0/3529628532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04ac148470 0x7f04ac1e01e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f049c01a0d0 tx=0x7f049c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:22.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04a2ffd640 1 -- 172.21.15.17:0/3529628532 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c0b0040 con 0x7f04ac148470 2024-03-16T10:11:22.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04a2ffd640 1 -- 172.21.15.17:0/3529628532 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f049c0b9070 con 0x7f04ac148470 2024-03-16T10:11:22.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04b0c78640 1 -- 172.21.15.17:0/3529628532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04ac14b1c0 con 0x7f04ac148470 2024-03-16T10:11:22.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04a2ffd640 1 -- 172.21.15.17:0/3529628532 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f049c0b4650 con 0x7f04ac148470 2024-03-16T10:11:22.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04b0c78640 1 -- 172.21.15.17:0/3529628532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04ac0b1de0 con 0x7f04ac148470 2024-03-16T10:11:22.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04b0c78640 1 -- 172.21.15.17:0/3529628532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04ac06a130 con 0x7f04ac148470 2024-03-16T10:11:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04a2ffd640 1 -- 172.21.15.17:0/3529628532 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f049c0b47f0 con 0x7f04ac148470 2024-03-16T10:11:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04a2ffd640 1 --2- 172.21.15.17:0/3529628532 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f048c067660 0x7f048c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04a2ffd640 1 -- 172.21.15.17:0/3529628532 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f049c11fe60 con 0x7f04ac148470 2024-03-16T10:11:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.884+0000 7f04aa59c640 1 --2- 172.21.15.17:0/3529628532 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f048c067660 0x7f048c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:22.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.888+0000 7f04aa59c640 1 --2- 172.21.15.17:0/3529628532 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f048c067660 0x7f048c069b20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009bd0 tx=0x7f04a400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:22.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:22.888+0000 7f04a2ffd640 1 -- 172.21.15.17:0/3529628532 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f049c14c6d0 con 0x7f04ac148470 2024-03-16T10:11:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.048+0000 7f04b0c78640 1 -- 172.21.15.17:0/3529628532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f04ac0b3b70 con 0x7f04ac148470 2024-03-16T10:11:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.048+0000 7f04a2ffd640 1 -- 172.21.15.17:0/3529628532 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f049c016090 con 0x7f04ac148470 2024-03-16T10:11:23.055 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-16T10:11:23.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.052+0000 7f04b0c78640 1 -- 172.21.15.17:0/3529628532 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f048c067660 msgr2=0x7f048c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.052+0000 7f04b0c78640 1 --2- 172.21.15.17:0/3529628532 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f048c067660 0x7f048c069b20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f04a4009bd0 tx=0x7f04a400b040 comp rx=0 tx=0).stop 2024-03-16T10:11:23.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.052+0000 7f04b0c78640 1 -- 172.21.15.17:0/3529628532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04ac148470 msgr2=0x7f04ac1e01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.052+0000 7f04b0c78640 1 --2- 172.21.15.17:0/3529628532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04ac148470 0x7f04ac1e01e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f049c01a0d0 tx=0x7f049c019240 comp rx=0 tx=0).stop 2024-03-16T10:11:23.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.052+0000 7f04b0c78640 1 -- 172.21.15.17:0/3529628532 shutdown_connections 2024-03-16T10:11:23.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.052+0000 7f04b0c78640 1 --2- 172.21.15.17:0/3529628532 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f048c067660 0x7f048c069b20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.052+0000 7f04b0c78640 1 --2- 172.21.15.17:0/3529628532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04ac148470 0x7f04ac1e01e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.052+0000 7f04b0c78640 1 --2- 172.21.15.17:0/3529628532 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f04ac0af8d0 0x7f04ac13eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.052+0000 7f04b0c78640 1 --2- 172.21.15.17:0/3529628532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f04ac06e3c0 0x7f04ac0b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.052+0000 7f04b0c78640 1 -- 172.21.15.17:0/3529628532 >> 172.21.15.17:0/3529628532 conn(0x7f04ac0b65d0 msgr2=0x7f04ac0726e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:23.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.052+0000 7f04b0c78640 1 -- 172.21.15.17:0/3529628532 shutdown_connections 2024-03-16T10:11:23.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.052+0000 7f04b0c78640 1 -- 172.21.15.17:0/3529628532 wait complete. 2024-03-16T10:11:23.074 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T10:11:23.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.184+0000 7feb2ef9f640 1 Processor -- start 2024-03-16T10:11:23.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.184+0000 7feb2ef9f640 1 -- start start 2024-03-16T10:11:23.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.184+0000 7feb2ef9f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feb280bd760 0x7feb281411c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.184+0000 7feb2ef9f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb280bdb40 con 0x7feb280b94d0 2024-03-16T10:11:23.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.184+0000 7feb2ef9f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb280b4630 con 0x7feb2814b2d0 2024-03-16T10:11:23.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.184+0000 7feb2ef9f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7feb280b4910 con 0x7feb280bd760 2024-03-16T10:11:23.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.184+0000 7feb2cd3b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7feb280bd760 0x7feb281411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.184+0000 7feb27fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7feb280b94d0 0x7feb2814abc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46148/0 (socket says 172.21.15.17:46148) 2024-03-16T10:11:23.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.184+0000 7feb27fff640 1 -- 172.21.15.17:0/3986064503 learned_addr learned my addr 172.21.15.17:0/3986064503 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:23.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.184+0000 7feb2cd3b640 1 -- 172.21.15.17:0/3986064503 >> v1:172.21.15.174:6789/0 conn(0x7feb2814b2d0 legacy=0x7feb28151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:23.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.184+0000 7feb2cd3b640 1 -- 172.21.15.17:0/3986064503 >> v1:172.21.15.17:6789/0 conn(0x7feb280b94d0 legacy=0x7feb2814abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.184+0000 7feb2cd3b640 1 -- 172.21.15.17:0/3986064503 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb280b55b0 con 0x7feb280bd760 2024-03-16T10:11:23.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.184+0000 7feb277fe640 1 -- 172.21.15.17:0/3986064503 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3121547836 0 0) 0x7feb280bdb40 con 0x7feb280b94d0 2024-03-16T10:11:23.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2cd3b640 1 --2- 172.21.15.17:0/3986064503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb280bd760 0x7feb281411c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feb280b57e0 tx=0x7feb1801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f99cccdf64937a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:23.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb277fe640 1 -- 172.21.15.17:0/3986064503 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb180b0070 con 0x7feb280bd760 2024-03-16T10:11:23.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb277fe640 1 -- 172.21.15.17:0/3986064503 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb180ab600 con 0x7feb280bd760 2024-03-16T10:11:23.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb277fe640 1 -- 172.21.15.17:0/3986064503 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb180ab9d0 con 0x7feb280bd760 2024-03-16T10:11:23.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 -- 172.21.15.17:0/3986064503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb280bd760 msgr2=0x7feb281411c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 --2- 172.21.15.17:0/3986064503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb280bd760 0x7feb281411c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feb280b57e0 tx=0x7feb1801c340 comp rx=0 tx=0).stop 2024-03-16T10:11:23.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 -- 172.21.15.17:0/3986064503 shutdown_connections 2024-03-16T10:11:23.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 --2- 172.21.15.17:0/3986064503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb280bd760 0x7feb281411c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 -- 172.21.15.17:0/3986064503 >> 172.21.15.17:0/3986064503 conn(0x7feb280b9f20 msgr2=0x7feb280bcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:23.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 -- 172.21.15.17:0/3986064503 shutdown_connections 2024-03-16T10:11:23.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 -- 172.21.15.17:0/3986064503 wait complete. 2024-03-16T10:11:23.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 Processor -- start 2024-03-16T10:11:23.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 -- start start 2024-03-16T10:11:23.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb280bd760 0x7feb2813ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb2814b2d0 0x7feb281e0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb281e0950 0x7feb281e3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7feb28155e60 con 0x7feb280bd760 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb28155ce0 con 0x7feb281e0950 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7feb28155fe0 con 0x7feb2814b2d0 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2cd3b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb280bd760 0x7feb2813ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2cd3b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb280bd760 0x7feb2813ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55322/0 (socket says 172.21.15.17:55322) 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2cd3b640 1 -- 172.21.15.17:0/1743154577 learned_addr learned my addr 172.21.15.17:0/1743154577 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2cd3b640 1 -- 172.21.15.17:0/1743154577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb2814b2d0 msgr2=0x7feb281e0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2cd3b640 1 --2- 172.21.15.17:0/1743154577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb2814b2d0 0x7feb281e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2cd3b640 1 -- 172.21.15.17:0/1743154577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb281e0950 msgr2=0x7feb281e3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2d53c640 1 --2- 172.21.15.17:0/1743154577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb281e0950 0x7feb281e3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2cd3b640 1 --2- 172.21.15.17:0/1743154577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb281e0950 0x7feb281e3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2cd3b640 1 -- 172.21.15.17:0/1743154577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb28148200 con 0x7feb280bd760 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2d53c640 1 --2- 172.21.15.17:0/1743154577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb281e0950 0x7feb281e3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2cd3b640 1 --2- 172.21.15.17:0/1743154577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb280bd760 0x7feb2813ab00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feb18019f70 tx=0x7feb1801ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb257fa640 1 -- 172.21.15.17:0/1743154577 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb180b0040 con 0x7feb280bd760 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb257fa640 1 -- 172.21.15.17:0/1743154577 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb18004230 con 0x7feb280bd760 2024-03-16T10:11:23.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 -- 172.21.15.17:0/1743154577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb280b3660 con 0x7feb280bd760 2024-03-16T10:11:23.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb257fa640 1 -- 172.21.15.17:0/1743154577 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb18003620 con 0x7feb280bd760 2024-03-16T10:11:23.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.188+0000 7feb2ef9f640 1 -- 172.21.15.17:0/1743154577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb281396d0 con 0x7feb280bd760 2024-03-16T10:11:23.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.192+0000 7feb2ef9f640 1 -- 172.21.15.17:0/1743154577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb2806a130 con 0x7feb280bd760 2024-03-16T10:11:23.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.192+0000 7feb257fa640 1 -- 172.21.15.17:0/1743154577 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7feb180037c0 con 0x7feb280bd760 2024-03-16T10:11:23.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.192+0000 7feb257fa640 1 --2- 172.21.15.17:0/1743154577 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feb14067660 0x7feb14069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.192+0000 7feb257fa640 1 -- 172.21.15.17:0/1743154577 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feb18120500 con 0x7feb280bd760 2024-03-16T10:11:23.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.192+0000 7feb27fff640 1 --2- 172.21.15.17:0/1743154577 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feb14067660 0x7feb14069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.192+0000 7feb27fff640 1 --2- 172.21.15.17:0/1743154577 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feb14067660 0x7feb14069b20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb28139150 tx=0x7feb1c007970 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:23.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.196+0000 7feb257fa640 1 -- 172.21.15.17:0/1743154577 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7feb18016050 con 0x7feb280bd760 2024-03-16T10:11:23.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.368+0000 7feb2ef9f640 1 -- 172.21.15.17:0/1743154577 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feb280bc760 con 0x7feb280bd760 2024-03-16T10:11:23.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.368+0000 7feb257fa640 1 -- 172.21.15.17:0/1743154577 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7feb180f1330 con 0x7feb280bd760 2024-03-16T10:11:23.374 INFO:teuthology.orchestra.run.smithi017.stderr:listed 0 entries 2024-03-16T10:11:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.372+0000 7feb2ef9f640 1 -- 172.21.15.17:0/1743154577 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feb14067660 msgr2=0x7feb14069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.372+0000 7feb2ef9f640 1 --2- 172.21.15.17:0/1743154577 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feb14067660 0x7feb14069b20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feb28139150 tx=0x7feb1c007970 comp rx=0 tx=0).stop 2024-03-16T10:11:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.372+0000 7feb2ef9f640 1 -- 172.21.15.17:0/1743154577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb280bd760 msgr2=0x7feb2813ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.372+0000 7feb2ef9f640 1 --2- 172.21.15.17:0/1743154577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb280bd760 0x7feb2813ab00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7feb18019f70 tx=0x7feb1801ab80 comp rx=0 tx=0).stop 2024-03-16T10:11:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.372+0000 7feb2ef9f640 1 -- 172.21.15.17:0/1743154577 shutdown_connections 2024-03-16T10:11:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.372+0000 7feb2ef9f640 1 --2- 172.21.15.17:0/1743154577 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feb14067660 0x7feb14069b20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.372+0000 7feb2ef9f640 1 --2- 172.21.15.17:0/1743154577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb281e0950 0x7feb281e3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.372+0000 7feb2ef9f640 1 --2- 172.21.15.17:0/1743154577 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7feb2814b2d0 0x7feb281e0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.372+0000 7feb2ef9f640 1 --2- 172.21.15.17:0/1743154577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb280bd760 0x7feb2813ab00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.372+0000 7feb2ef9f640 1 -- 172.21.15.17:0/1743154577 >> 172.21.15.17:0/1743154577 conn(0x7feb280b9f20 msgr2=0x7feb28137bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.372+0000 7feb2ef9f640 1 -- 172.21.15.17:0/1743154577 shutdown_connections 2024-03-16T10:11:23.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.372+0000 7feb2ef9f640 1 -- 172.21.15.17:0/1743154577 wait complete. 2024-03-16T10:11:23.392 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:11:23.392 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T10:11:23.398 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:23.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.548+0000 7f325b503640 1 Processor -- start 2024-03-16T10:11:23.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.548+0000 7f325b503640 1 -- start start 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.548+0000 7f325b503640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3254067220 0x7f325409bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.548+0000 7f325b503640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32541414c0 0x7f32541474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.548+0000 7f325b503640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3254141940 0x7f3254147a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.548+0000 7f325b503640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f32540b4ae0 con 0x7f3254141940 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.548+0000 7f325b503640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f3254067600 con 0x7f3254067220 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.548+0000 7f325b503640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f32540b40f0 con 0x7f32541414c0 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f3259aa0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3254141940 0x7f3254147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f3259aa0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3254141940 0x7f3254147a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55332/0 (socket says 172.21.15.17:55332) 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f3259aa0640 1 -- 172.21.15.17:0/2640279941 learned_addr learned my addr 172.21.15.17:0/2640279941 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f3258a9e640 1 --2- 172.21.15.17:0/2640279941 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32541414c0 0x7f32541474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f325929f640 1 --2- 172.21.15.17:0/2640279941 >> v2:172.21.15.174:3300/0 conn(0x7f3254067220 0x7f325409bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f3259aa0640 1 -- 172.21.15.17:0/2640279941 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32541414c0 msgr2=0x7f32541474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f3259aa0640 1 --2- 172.21.15.17:0/2640279941 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32541414c0 0x7f32541474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f3259aa0640 1 -- 172.21.15.17:0/2640279941 >> v2:172.21.15.174:3300/0 conn(0x7f3254067220 msgr2=0x7f325409bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:23.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f3259aa0640 1 --2- 172.21.15.17:0/2640279941 >> v2:172.21.15.174:3300/0 conn(0x7f3254067220 0x7f325409bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f3259aa0640 1 -- 172.21.15.17:0/2640279941 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32540b4770 con 0x7f3254141940 2024-03-16T10:11:23.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f3259aa0640 1 --2- 172.21.15.17:0/2640279941 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3254141940 0x7f3254147a30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3254070970 tx=0x7f324c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205d9f66c3a22ba0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f324bfff640 1 -- 172.21.15.17:0/2640279941 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c0a55e0 con 0x7f3254141940 2024-03-16T10:11:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f324bfff640 1 -- 172.21.15.17:0/2640279941 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f324c0a5780 con 0x7f3254141940 2024-03-16T10:11:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f324bfff640 1 -- 172.21.15.17:0/2640279941 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324c0a5b50 con 0x7f3254141940 2024-03-16T10:11:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f325b503640 1 -- 172.21.15.17:0/2640279941 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3254141940 msgr2=0x7f3254147a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f325b503640 1 --2- 172.21.15.17:0/2640279941 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3254141940 0x7f3254147a30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3254070970 tx=0x7f324c01c960 comp rx=0 tx=0).stop 2024-03-16T10:11:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f325b503640 1 -- 172.21.15.17:0/2640279941 shutdown_connections 2024-03-16T10:11:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f325b503640 1 --2- 172.21.15.17:0/2640279941 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3254141940 0x7f3254147a30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f325b503640 1 --2- 172.21.15.17:0/2640279941 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f32541414c0 0x7f32541474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f325b503640 1 --2- 172.21.15.17:0/2640279941 >> v2:172.21.15.174:3300/0 conn(0x7f3254067220 0x7f325409bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f325b503640 1 -- 172.21.15.17:0/2640279941 >> 172.21.15.17:0/2640279941 conn(0x7f32540b5e10 msgr2=0x7f32540b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f325b503640 1 -- 172.21.15.17:0/2640279941 shutdown_connections 2024-03-16T10:11:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.552+0000 7f325b503640 1 -- 172.21.15.17:0/2640279941 wait complete. 2024-03-16T10:11:23.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f325b503640 1 Processor -- start 2024-03-16T10:11:23.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f325b503640 1 -- start start 2024-03-16T10:11:23.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f325b503640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3254067220 0x7f3254145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f325b503640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32541414c0 0x7f3254145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f325b503640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3254141940 0x7f32541dfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f325b503640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f325414b500 con 0x7f32541414c0 2024-03-16T10:11:23.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f325b503640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f325414b380 con 0x7f3254141940 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f325b503640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f325414b680 con 0x7f3254067220 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f3258a9e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32541414c0 0x7f3254145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f3258a9e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32541414c0 0x7f3254145ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55344/0 (socket says 172.21.15.17:55344) 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f3258a9e640 1 -- 172.21.15.17:0/938717616 learned_addr learned my addr 172.21.15.17:0/938717616 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f3259aa0640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3254141940 0x7f32541dfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f325929f640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3254067220 0x7f3254145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f3258a9e640 1 -- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3254067220 msgr2=0x7f3254145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f3258a9e640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3254067220 0x7f3254145260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f3258a9e640 1 -- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3254141940 msgr2=0x7f32541dfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f3258a9e640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3254141940 0x7f32541dfec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f3258a9e640 1 -- 172.21.15.17:0/938717616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32541dfa90 con 0x7f32541414c0 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f325929f640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3254067220 0x7f3254145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f3258a9e640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32541414c0 0x7f3254145ca0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f324400d800 tx=0x7f324400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f3249ffb640 1 -- 172.21.15.17:0/938717616 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324400b6c0 con 0x7f32541414c0 2024-03-16T10:11:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f3249ffb640 1 -- 172.21.15.17:0/938717616 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3244019070 con 0x7f32541414c0 2024-03-16T10:11:23.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f3249ffb640 1 -- 172.21.15.17:0/938717616 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f324400ba60 con 0x7f32541414c0 2024-03-16T10:11:23.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f325b503640 1 -- 172.21.15.17:0/938717616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3254073a80 con 0x7f32541414c0 2024-03-16T10:11:23.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f325b503640 1 -- 172.21.15.17:0/938717616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3254143c50 con 0x7f32541414c0 2024-03-16T10:11:23.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.556+0000 7f325b503640 1 -- 172.21.15.17:0/938717616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f325406a130 con 0x7f32541414c0 2024-03-16T10:11:23.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.560+0000 7f3249ffb640 1 -- 172.21.15.17:0/938717616 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f324400bc00 con 0x7f32541414c0 2024-03-16T10:11:23.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.560+0000 7f3249ffb640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3238067660 0x7f3238069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.560+0000 7f3249ffb640 1 -- 172.21.15.17:0/938717616 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3244085c70 con 0x7f32541414c0 2024-03-16T10:11:23.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.560+0000 7f325929f640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3238067660 0x7f3238069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.560+0000 7f325929f640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3238067660 0x7f3238069b20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3250009840 tx=0x7f3250008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:23.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.568+0000 7f3249ffb640 1 -- 172.21.15.17:0/938717616 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3244056980 con 0x7f32541414c0 2024-03-16T10:11:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.732+0000 7f325b503640 1 -- 172.21.15.17:0/938717616 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32540ae3d0 con 0x7f32541414c0 2024-03-16T10:11:23.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.732+0000 7f3249ffb640 1 -- 172.21.15.17:0/938717616 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7f3244056280 con 0x7f32541414c0 2024-03-16T10:11:23.739 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:23.739 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.736+0000 7f325b503640 1 -- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3238067660 msgr2=0x7f3238069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.736+0000 7f325b503640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3238067660 0x7f3238069b20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3250009840 tx=0x7f3250008040 comp rx=0 tx=0).stop 2024-03-16T10:11:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.736+0000 7f325b503640 1 -- 172.21.15.17:0/938717616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32541414c0 msgr2=0x7f3254145ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.736+0000 7f325b503640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32541414c0 0x7f3254145ca0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f324400d800 tx=0x7f324400dcc0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.736+0000 7f325b503640 1 -- 172.21.15.17:0/938717616 shutdown_connections 2024-03-16T10:11:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.736+0000 7f325b503640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3238067660 0x7f3238069b20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.736+0000 7f325b503640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3254141940 0x7f32541dfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.736+0000 7f325b503640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32541414c0 0x7f3254145ca0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.736+0000 7f325b503640 1 --2- 172.21.15.17:0/938717616 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3254067220 0x7f3254145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.736+0000 7f325b503640 1 -- 172.21.15.17:0/938717616 >> 172.21.15.17:0/938717616 conn(0x7f32540b5e10 msgr2=0x7f32540b7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.736+0000 7f325b503640 1 -- 172.21.15.17:0/938717616 shutdown_connections 2024-03-16T10:11:23.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.736+0000 7f325b503640 1 -- 172.21.15.17:0/938717616 wait complete. 2024-03-16T10:11:23.758 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:10:55.723734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:10:56.459894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:11:23.759 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:11:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc4636eb640 1 Processor -- start 2024-03-16T10:11:23.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc4636eb640 1 -- start start 2024-03-16T10:11:23.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc4636eb640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc45c067220 0x7fc45c1488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc4636eb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc45c09bd00 0x7fc45c09d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc4636eb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc45c148df0 0x7fc45c09daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc4636eb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc45c09e820 con 0x7fc45c09bd00 2024-03-16T10:11:23.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc4636eb640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fc45c0b42b0 con 0x7fc45c067220 2024-03-16T10:11:23.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc4636eb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc45c067600 con 0x7fc45c148df0 2024-03-16T10:11:23.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc460c86640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc45c09bd00 0x7fc45c09d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc460c86640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc45c09bd00 0x7fc45c09d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55356/0 (socket says 172.21.15.17:55356) 2024-03-16T10:11:23.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc460c86640 1 -- 172.21.15.17:0/2756112201 learned_addr learned my addr 172.21.15.17:0/2756112201 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:23.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc461c88640 1 --2- 172.21.15.17:0/2756112201 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc45c148df0 0x7fc45c09daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc461487640 1 --2- 172.21.15.17:0/2756112201 >> v2:172.21.15.174:3300/0 conn(0x7fc45c067220 0x7fc45c1488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc460c86640 1 -- 172.21.15.17:0/2756112201 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc45c148df0 msgr2=0x7fc45c09daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:23.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc460c86640 1 --2- 172.21.15.17:0/2756112201 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc45c148df0 0x7fc45c09daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc460c86640 1 -- 172.21.15.17:0/2756112201 >> v2:172.21.15.174:3300/0 conn(0x7fc45c067220 msgr2=0x7fc45c1488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:23.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc460c86640 1 --2- 172.21.15.17:0/2756112201 >> v2:172.21.15.174:3300/0 conn(0x7fc45c067220 0x7fc45c1488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc460c86640 1 -- 172.21.15.17:0/2756112201 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45c0b55d0 con 0x7fc45c09bd00 2024-03-16T10:11:23.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.884+0000 7fc461487640 1 --2- 172.21.15.17:0/2756112201 >> v2:172.21.15.174:3300/0 conn(0x7fc45c067220 0x7fc45c1488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:23.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc460c86640 1 --2- 172.21.15.17:0/2756112201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc45c09bd00 0x7fc45c09d560 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0b0580 tx=0x7fc44c01cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16f73cd327febd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:23.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc453fff640 1 -- 172.21.15.17:0/2756112201 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0ab640 con 0x7fc45c09bd00 2024-03-16T10:11:23.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc453fff640 1 -- 172.21.15.17:0/2756112201 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc44c0b0070 con 0x7fc45c09bd00 2024-03-16T10:11:23.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc453fff640 1 -- 172.21.15.17:0/2756112201 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44c0aba00 con 0x7fc45c09bd00 2024-03-16T10:11:23.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 -- 172.21.15.17:0/2756112201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc45c09bd00 msgr2=0x7fc45c09d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 --2- 172.21.15.17:0/2756112201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc45c09bd00 0x7fc45c09d560 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc45c0b0580 tx=0x7fc44c01cec0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 -- 172.21.15.17:0/2756112201 shutdown_connections 2024-03-16T10:11:23.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 --2- 172.21.15.17:0/2756112201 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc45c148df0 0x7fc45c09daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 --2- 172.21.15.17:0/2756112201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc45c09bd00 0x7fc45c09d560 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 --2- 172.21.15.17:0/2756112201 >> v2:172.21.15.174:3300/0 conn(0x7fc45c067220 0x7fc45c1488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 -- 172.21.15.17:0/2756112201 >> 172.21.15.17:0/2756112201 conn(0x7fc45c06bd70 msgr2=0x7fc45c0b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 -- 172.21.15.17:0/2756112201 shutdown_connections 2024-03-16T10:11:23.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 -- 172.21.15.17:0/2756112201 wait complete. 2024-03-16T10:11:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 Processor -- start 2024-03-16T10:11:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 -- start start 2024-03-16T10:11:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc45c067220 0x7fc45c0aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc45c141aa0 0x7fc45c14f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc45c13c3f0 0x7fc45c13c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c09ad80 con 0x7fc45c13c3f0 2024-03-16T10:11:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc45c09e0d0 con 0x7fc45c141aa0 2024-03-16T10:11:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc4636eb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc45c09af00 con 0x7fc45c067220 2024-03-16T10:11:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc461c88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc45c13c3f0 0x7fc45c13c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc461c88640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc45c13c3f0 0x7fc45c13c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55364/0 (socket says 172.21.15.17:55364) 2024-03-16T10:11:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc461c88640 1 -- 172.21.15.17:0/1282959576 learned_addr learned my addr 172.21.15.17:0/1282959576 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:23.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc461487640 1 --2- 172.21.15.17:0/1282959576 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc45c067220 0x7fc45c0aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc460c86640 1 --2- 172.21.15.17:0/1282959576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc45c141aa0 0x7fc45c14f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc461c88640 1 -- 172.21.15.17:0/1282959576 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc45c067220 msgr2=0x7fc45c0aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc461c88640 1 --2- 172.21.15.17:0/1282959576 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc45c067220 0x7fc45c0aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc461c88640 1 -- 172.21.15.17:0/1282959576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc45c141aa0 msgr2=0x7fc45c14f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc461c88640 1 --2- 172.21.15.17:0/1282959576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc45c141aa0 0x7fc45c14f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.888+0000 7fc461c88640 1 -- 172.21.15.17:0/1282959576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45c14f740 con 0x7fc45c13c3f0 2024-03-16T10:11:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.892+0000 7fc461c88640 1 --2- 172.21.15.17:0/1282959576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc45c13c3f0 0x7fc45c13c880 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc45800d500 tx=0x7fc45800d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.892+0000 7fc451ffb640 1 -- 172.21.15.17:0/1282959576 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4580125f0 con 0x7fc45c13c3f0 2024-03-16T10:11:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.892+0000 7fc451ffb640 1 -- 172.21.15.17:0/1282959576 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc458019070 con 0x7fc45c13c3f0 2024-03-16T10:11:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.892+0000 7fc451ffb640 1 -- 172.21.15.17:0/1282959576 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4580129d0 con 0x7fc45c13c3f0 2024-03-16T10:11:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.892+0000 7fc4636eb640 1 -- 172.21.15.17:0/1282959576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45c143b20 con 0x7fc45c13c3f0 2024-03-16T10:11:23.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.892+0000 7fc4636eb640 1 -- 172.21.15.17:0/1282959576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc45c0af230 con 0x7fc45c13c3f0 2024-03-16T10:11:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.892+0000 7fc4636eb640 1 -- 172.21.15.17:0/1282959576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc45c06a130 con 0x7fc45c13c3f0 2024-03-16T10:11:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.892+0000 7fc451ffb640 1 -- 172.21.15.17:0/1282959576 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fc458012b70 con 0x7fc45c13c3f0 2024-03-16T10:11:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.892+0000 7fc451ffb640 1 --2- 172.21.15.17:0/1282959576 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc440067660 0x7fc440069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:23.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.892+0000 7fc451ffb640 1 -- 172.21.15.17:0/1282959576 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc458086620 con 0x7fc45c13c3f0 2024-03-16T10:11:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.892+0000 7fc461487640 1 --2- 172.21.15.17:0/1282959576 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc440067660 0x7fc440069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:23.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.896+0000 7fc461487640 1 --2- 172.21.15.17:0/1282959576 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc440067660 0x7fc440069b20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc45c14fb70 tx=0x7fc4540039a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:23.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:23.896+0000 7fc451ffb640 1 -- 172.21.15.17:0/1282959576 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc458057330 con 0x7fc45c13c3f0 2024-03-16T10:11:24.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.056+0000 7fc4636eb640 1 -- 172.21.15.17:0/1282959576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc45c0b7340 con 0x7fc45c13c3f0 2024-03-16T10:11:24.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.056+0000 7fc451ffb640 1 -- 172.21.15.17:0/1282959576 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fc458056c30 con 0x7fc45c13c3f0 2024-03-16T10:11:24.062 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:11:24.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.060+0000 7fc4636eb640 1 -- 172.21.15.17:0/1282959576 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc440067660 msgr2=0x7fc440069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.060+0000 7fc4636eb640 1 --2- 172.21.15.17:0/1282959576 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc440067660 0x7fc440069b20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc45c14fb70 tx=0x7fc4540039a0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.060+0000 7fc4636eb640 1 -- 172.21.15.17:0/1282959576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc45c13c3f0 msgr2=0x7fc45c13c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.060+0000 7fc4636eb640 1 --2- 172.21.15.17:0/1282959576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc45c13c3f0 0x7fc45c13c880 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc45800d500 tx=0x7fc45800d9c0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.060+0000 7fc4636eb640 1 -- 172.21.15.17:0/1282959576 shutdown_connections 2024-03-16T10:11:24.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.060+0000 7fc4636eb640 1 --2- 172.21.15.17:0/1282959576 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc440067660 0x7fc440069b20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.060+0000 7fc4636eb640 1 --2- 172.21.15.17:0/1282959576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc45c13c3f0 0x7fc45c13c880 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.060+0000 7fc4636eb640 1 --2- 172.21.15.17:0/1282959576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc45c141aa0 0x7fc45c14f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.060+0000 7fc4636eb640 1 --2- 172.21.15.17:0/1282959576 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc45c067220 0x7fc45c0aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.060+0000 7fc4636eb640 1 -- 172.21.15.17:0/1282959576 >> 172.21.15.17:0/1282959576 conn(0x7fc45c06bd70 msgr2=0x7fc45c143070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:24.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.060+0000 7fc4636eb640 1 -- 172.21.15.17:0/1282959576 shutdown_connections 2024-03-16T10:11:24.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.060+0000 7fc4636eb640 1 -- 172.21.15.17:0/1282959576 wait complete. 2024-03-16T10:11:24.088 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:11:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.220+0000 7f2d72dfa640 1 Processor -- start 2024-03-16T10:11:24.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.220+0000 7f2d72dfa640 1 -- start start 2024-03-16T10:11:24.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.220+0000 7f2d72dfa640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2d6c0b4800 0x7f2d6c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.220+0000 7f2d72dfa640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d6c0b1ac0 con 0x7f2d6c0b1f70 2024-03-16T10:11:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.220+0000 7f2d72dfa640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d6c1487e0 con 0x7f2d6c147ca0 2024-03-16T10:11:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.220+0000 7f2d72dfa640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f2d6c0b4be0 con 0x7f2d6c0b4800 2024-03-16T10:11:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.220+0000 7f2d70b96640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2d6c0b4800 0x7f2d6c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.220+0000 7f2d63fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2d6c0b1f70 0x7f2d6c0b23b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46158/0 (socket says 172.21.15.17:46158) 2024-03-16T10:11:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.220+0000 7f2d70b96640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2d6c0b4800 0x7f2d6c0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55370/0 (socket says 172.21.15.17:55370) 2024-03-16T10:11:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.220+0000 7f2d63fff640 1 -- 172.21.15.17:0/1794035967 learned_addr learned my addr 172.21.15.17:0/1794035967 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.220+0000 7f2d70b96640 1 -- 172.21.15.17:0/1794035967 >> v1:172.21.15.174:6789/0 conn(0x7f2d6c147ca0 legacy=0x7f2d6c1480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.220+0000 7f2d70b96640 1 -- 172.21.15.17:0/1794035967 >> v1:172.21.15.17:6789/0 conn(0x7f2d6c0b1f70 legacy=0x7f2d6c0b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.220+0000 7f2d70b96640 1 -- 172.21.15.17:0/1794035967 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6c0b1220 con 0x7f2d6c0b4800 2024-03-16T10:11:24.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.220+0000 7f2d637fe640 1 -- 172.21.15.17:0/1794035967 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1600266702 0 0) 0x7f2d6c0b1ac0 con 0x7f2d6c0b1f70 2024-03-16T10:11:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.224+0000 7f2d70b96640 1 --2- 172.21.15.17:0/1794035967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d6c0b4800 0x7f2d6c0b0bc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2d680097c0 tx=0x7f2d6801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de15dd2d62867e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.224+0000 7f2d637fe640 1 -- 172.21.15.17:0/1794035967 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d680b0070 con 0x7f2d6c0b4800 2024-03-16T10:11:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.224+0000 7f2d637fe640 1 -- 172.21.15.17:0/1794035967 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d680ab600 con 0x7f2d6c0b4800 2024-03-16T10:11:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.224+0000 7f2d637fe640 1 -- 172.21.15.17:0/1794035967 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d680ab9d0 con 0x7f2d6c0b4800 2024-03-16T10:11:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.224+0000 7f2d72dfa640 1 -- 172.21.15.17:0/1794035967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d6c0b4800 msgr2=0x7f2d6c0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.224+0000 7f2d72dfa640 1 --2- 172.21.15.17:0/1794035967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d6c0b4800 0x7f2d6c0b0bc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2d680097c0 tx=0x7f2d6801c410 comp rx=0 tx=0).stop 2024-03-16T10:11:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.224+0000 7f2d72dfa640 1 -- 172.21.15.17:0/1794035967 shutdown_connections 2024-03-16T10:11:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.224+0000 7f2d72dfa640 1 --2- 172.21.15.17:0/1794035967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d6c0b4800 0x7f2d6c0b0bc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.224+0000 7f2d72dfa640 1 -- 172.21.15.17:0/1794035967 >> 172.21.15.17:0/1794035967 conn(0x7f2d6c0b8e90 msgr2=0x7f2d6c0b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.224+0000 7f2d72dfa640 1 -- 172.21.15.17:0/1794035967 shutdown_connections 2024-03-16T10:11:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.224+0000 7f2d72dfa640 1 -- 172.21.15.17:0/1794035967 wait complete. 2024-03-16T10:11:24.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d72dfa640 1 Processor -- start 2024-03-16T10:11:24.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d72dfa640 1 -- start start 2024-03-16T10:11:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d72dfa640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d6c0b1f70 0x7f2d6c09d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d72dfa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d6c0b4800 0x7f2d6c09af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d72dfa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d6c147ca0 0x7f2d6c146840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d72dfa640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1525c0 con 0x7f2d6c147ca0 2024-03-16T10:11:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d72dfa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c152440 con 0x7f2d6c0b4800 2024-03-16T10:11:24.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d72dfa640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2d6c152740 con 0x7f2d6c0b1f70 2024-03-16T10:11:24.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d71397640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d6c147ca0 0x7f2d6c146840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d71397640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d6c147ca0 0x7f2d6c146840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55384/0 (socket says 172.21.15.17:55384) 2024-03-16T10:11:24.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d71397640 1 -- 172.21.15.17:0/683887167 learned_addr learned my addr 172.21.15.17:0/683887167 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:24.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d70b96640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d6c0b1f70 0x7f2d6c09d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d63fff640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d6c0b4800 0x7f2d6c09af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d71397640 1 -- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d6c0b1f70 msgr2=0x7f2d6c09d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d71397640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d6c0b1f70 0x7f2d6c09d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d71397640 1 -- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d6c0b4800 msgr2=0x7f2d6c09af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d71397640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d6c0b4800 0x7f2d6c09af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d71397640 1 -- 172.21.15.17:0/683887167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6c09db90 con 0x7f2d6c147ca0 2024-03-16T10:11:24.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d63fff640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d6c0b4800 0x7f2d6c09af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:24.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d71397640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d6c147ca0 0x7f2d6c146840 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400c730 tx=0x7f2d6400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:24.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d617fa640 1 -- 172.21.15.17:0/683887167 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64003b80 con 0x7f2d6c147ca0 2024-03-16T10:11:24.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d617fa640 1 -- 172.21.15.17:0/683887167 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d64003d20 con 0x7f2d6c147ca0 2024-03-16T10:11:24.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d72dfa640 1 -- 172.21.15.17:0/683887167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6c09de70 con 0x7f2d6c147ca0 2024-03-16T10:11:24.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d617fa640 1 -- 172.21.15.17:0/683887167 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400aa30 con 0x7f2d6c147ca0 2024-03-16T10:11:24.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d72dfa640 1 -- 172.21.15.17:0/683887167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6c1e1d00 con 0x7f2d6c147ca0 2024-03-16T10:11:24.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.228+0000 7f2d72dfa640 1 -- 172.21.15.17:0/683887167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d6c068360 con 0x7f2d6c147ca0 2024-03-16T10:11:24.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.232+0000 7f2d617fa640 1 -- 172.21.15.17:0/683887167 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f2d6400abd0 con 0x7f2d6c147ca0 2024-03-16T10:11:24.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.232+0000 7f2d617fa640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2d58067660 0x7f2d58069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.232+0000 7f2d617fa640 1 -- 172.21.15.17:0/683887167 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2d640856a0 con 0x7f2d6c147ca0 2024-03-16T10:11:24.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.232+0000 7f2d70b96640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2d58067660 0x7f2d58069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.232+0000 7f2d70b96640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2d58067660 0x7f2d58069b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d680ad000 tx=0x7f2d680a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:24.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.236+0000 7f2d617fa640 1 -- 172.21.15.17:0/683887167 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2d640563b0 con 0x7f2d6c147ca0 2024-03-16T10:11:24.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.396+0000 7f2d72dfa640 1 -- 172.21.15.17:0/683887167 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2d6c0b7db0 con 0x7f2d6c147ca0 2024-03-16T10:11:24.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.396+0000 7f2d617fa640 1 -- 172.21.15.17:0/683887167 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f2d64055cb0 con 0x7f2d6c147ca0 2024-03-16T10:11:24.402 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-16T10:11:24.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.400+0000 7f2d72dfa640 1 -- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2d58067660 msgr2=0x7f2d58069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.400+0000 7f2d72dfa640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2d58067660 0x7f2d58069b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2d680ad000 tx=0x7f2d680a6000 comp rx=0 tx=0).stop 2024-03-16T10:11:24.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.400+0000 7f2d72dfa640 1 -- 172.21.15.17:0/683887167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d6c147ca0 msgr2=0x7f2d6c146840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.400+0000 7f2d72dfa640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d6c147ca0 0x7f2d6c146840 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400c730 tx=0x7f2d6400cbf0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.400+0000 7f2d72dfa640 1 -- 172.21.15.17:0/683887167 shutdown_connections 2024-03-16T10:11:24.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.400+0000 7f2d72dfa640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2d58067660 0x7f2d58069b20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.400+0000 7f2d72dfa640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d6c147ca0 0x7f2d6c146840 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.400+0000 7f2d72dfa640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d6c0b4800 0x7f2d6c09af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.400+0000 7f2d72dfa640 1 --2- 172.21.15.17:0/683887167 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d6c0b1f70 0x7f2d6c09d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.400+0000 7f2d72dfa640 1 -- 172.21.15.17:0/683887167 >> 172.21.15.17:0/683887167 conn(0x7f2d6c0b8e90 msgr2=0x7f2d6c06cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:24.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.400+0000 7f2d72dfa640 1 -- 172.21.15.17:0/683887167 shutdown_connections 2024-03-16T10:11:24.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.400+0000 7f2d72dfa640 1 -- 172.21.15.17:0/683887167 wait complete. 2024-03-16T10:11:24.428 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:11:24.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.544+0000 7f3a014b6640 1 Processor -- start 2024-03-16T10:11:24.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.544+0000 7f3a014b6640 1 -- start start 2024-03-16T10:11:24.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f3a014b6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39fc14b230 0x7f39fc144480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f3a014b6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f39fc1449c0 0x7f39fc144e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f3a014b6640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39fc067600 con 0x7f39fc067220 2024-03-16T10:11:24.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f3a014b6640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f39fc0b46f0 con 0x7f39fc1449c0 2024-03-16T10:11:24.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f3a014b6640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f39fc0b57f0 con 0x7f39fc14b230 2024-03-16T10:11:24.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f39fb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f39fc1449c0 0x7f39fc144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f39fb7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f39fc1449c0 0x7f39fc144e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55400/0 (socket says 172.21.15.17:55400) 2024-03-16T10:11:24.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f39fb7fe640 1 -- 172.21.15.17:0/2083472498 learned_addr learned my addr 172.21.15.17:0/2083472498 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:24.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f39fa7fc640 1 --2- 172.21.15.17:0/2083472498 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39fc14b230 0x7f39fc144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f39fb7fe640 1 -- 172.21.15.17:0/2083472498 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39fc14b230 msgr2=0x7f39fc144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:24.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f39fb7fe640 1 --2- 172.21.15.17:0/2083472498 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39fc14b230 0x7f39fc144480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f39fb7fe640 1 -- 172.21.15.17:0/2083472498 >> v1:172.21.15.174:6789/0 conn(0x7f39fc067220 legacy=0x7f39fc14ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:24.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f39fb7fe640 1 -- 172.21.15.17:0/2083472498 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc0b4080 con 0x7f39fc1449c0 2024-03-16T10:11:24.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f39fa7fc640 1 --2- 172.21.15.17:0/2083472498 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39fc14b230 0x7f39fc144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:24.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f39fb7fe640 1 --2- 172.21.15.17:0/2083472498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39fc1449c0 0x7f39fc144e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0b4870 tx=0x7f39f401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa1b7b393c1aa02 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:24.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f39f9ffb640 1 -- 172.21.15.17:0/2083472498 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f40a55c0 con 0x7f39fc1449c0 2024-03-16T10:11:24.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f39f9ffb640 1 -- 172.21.15.17:0/2083472498 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39f40a5760 con 0x7f39fc1449c0 2024-03-16T10:11:24.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.548+0000 7f39f9ffb640 1 -- 172.21.15.17:0/2083472498 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f40a5b30 con 0x7f39fc1449c0 2024-03-16T10:11:24.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 -- 172.21.15.17:0/2083472498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39fc1449c0 msgr2=0x7f39fc144e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 --2- 172.21.15.17:0/2083472498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39fc1449c0 0x7f39fc144e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0b4870 tx=0x7f39f401c940 comp rx=0 tx=0).stop 2024-03-16T10:11:24.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 -- 172.21.15.17:0/2083472498 shutdown_connections 2024-03-16T10:11:24.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 --2- 172.21.15.17:0/2083472498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39fc1449c0 0x7f39fc144e10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 --2- 172.21.15.17:0/2083472498 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39fc14b230 0x7f39fc144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 -- 172.21.15.17:0/2083472498 >> 172.21.15.17:0/2083472498 conn(0x7f39fc06bd70 msgr2=0x7f39fc0b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:24.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 -- 172.21.15.17:0/2083472498 shutdown_connections 2024-03-16T10:11:24.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 -- 172.21.15.17:0/2083472498 wait complete. 2024-03-16T10:11:24.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 Processor -- start 2024-03-16T10:11:24.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 -- start start 2024-03-16T10:11:24.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39fc067220 0x7f39fc072520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39fc1449c0 0x7f39fc1dfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39fc14b230 0x7f39fc143bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc0b3e20 con 0x7f39fc067220 2024-03-16T10:11:24.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc0b3ca0 con 0x7f39fc14b230 2024-03-16T10:11:24.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f39fc154880 con 0x7f39fc1449c0 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39faffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39fc067220 0x7f39fc072520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39faffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39fc067220 0x7f39fc072520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55416/0 (socket says 172.21.15.17:55416) 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39faffd640 1 -- 172.21.15.17:0/3392438975 learned_addr learned my addr 172.21.15.17:0/3392438975 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39fa7fc640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39fc1449c0 0x7f39fc1dfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39fb7fe640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39fc14b230 0x7f39fc143bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39faffd640 1 -- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39fc1449c0 msgr2=0x7f39fc1dfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39faffd640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39fc1449c0 0x7f39fc1dfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39faffd640 1 -- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39fc14b230 msgr2=0x7f39fc143bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39faffd640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39fc14b230 0x7f39fc143bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39faffd640 1 -- 172.21.15.17:0/3392438975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc0b1ad0 con 0x7f39fc067220 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39fb7fe640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39fc14b230 0x7f39fc143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39fa7fc640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39fc1449c0 0x7f39fc1dfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39faffd640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39fc067220 0x7f39fc072520 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f39f000c480 tx=0x7f39f000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39e37fe640 1 -- 172.21.15.17:0/3392438975 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f0012830 con 0x7f39fc067220 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39e37fe640 1 -- 172.21.15.17:0/3392438975 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39f0017070 con 0x7f39fc067220 2024-03-16T10:11:24.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f39e37fe640 1 -- 172.21.15.17:0/3392438975 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f0012c30 con 0x7f39fc067220 2024-03-16T10:11:24.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 -- 172.21.15.17:0/3392438975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39fc147430 con 0x7f39fc067220 2024-03-16T10:11:24.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.552+0000 7f3a014b6640 1 -- 172.21.15.17:0/3392438975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39fc0b1d40 con 0x7f39fc067220 2024-03-16T10:11:24.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.556+0000 7f3a014b6640 1 -- 172.21.15.17:0/3392438975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39fc06a130 con 0x7f39fc067220 2024-03-16T10:11:24.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.556+0000 7f39e37fe640 1 -- 172.21.15.17:0/3392438975 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f39f001c460 con 0x7f39fc067220 2024-03-16T10:11:24.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.556+0000 7f39e37fe640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f39e8067660 0x7f39e8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.556+0000 7f39e37fe640 1 -- 172.21.15.17:0/3392438975 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f39f00855d0 con 0x7f39fc067220 2024-03-16T10:11:24.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.556+0000 7f39fa7fc640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f39e8067660 0x7f39e8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.556+0000 7f39fa7fc640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f39e8067660 0x7f39e8069b20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f39fc1dd870 tx=0x7f39ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:24.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.560+0000 7f39e37fe640 1 -- 172.21.15.17:0/3392438975 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f39f00562e0 con 0x7f39fc067220 2024-03-16T10:11:24.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.716+0000 7f3a014b6640 1 -- 172.21.15.17:0/3392438975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f39fc0b0810 con 0x7f39fc067220 2024-03-16T10:11:24.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.716+0000 7f39e37fe640 1 -- 172.21.15.17:0/3392438975 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f39f0055be0 con 0x7f39fc067220 2024-03-16T10:11:24.720 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-16T10:11:24.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.716+0000 7f3a014b6640 1 -- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f39e8067660 msgr2=0x7f39e8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.716+0000 7f3a014b6640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f39e8067660 0x7f39e8069b20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f39fc1dd870 tx=0x7f39ec008040 comp rx=0 tx=0).stop 2024-03-16T10:11:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.716+0000 7f3a014b6640 1 -- 172.21.15.17:0/3392438975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39fc067220 msgr2=0x7f39fc072520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.716+0000 7f3a014b6640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39fc067220 0x7f39fc072520 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f39f000c480 tx=0x7f39f000c940 comp rx=0 tx=0).stop 2024-03-16T10:11:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.716+0000 7f3a014b6640 1 -- 172.21.15.17:0/3392438975 shutdown_connections 2024-03-16T10:11:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.716+0000 7f3a014b6640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f39e8067660 0x7f39e8069b20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.716+0000 7f3a014b6640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39fc14b230 0x7f39fc143bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.716+0000 7f3a014b6640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f39fc1449c0 0x7f39fc1dfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.716+0000 7f3a014b6640 1 --2- 172.21.15.17:0/3392438975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f39fc067220 0x7f39fc072520 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.716+0000 7f3a014b6640 1 -- 172.21.15.17:0/3392438975 >> 172.21.15.17:0/3392438975 conn(0x7f39fc06bd70 msgr2=0x7f39fc149980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.716+0000 7f3a014b6640 1 -- 172.21.15.17:0/3392438975 shutdown_connections 2024-03-16T10:11:24.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.716+0000 7f3a014b6640 1 -- 172.21.15.17:0/3392438975 wait complete. 2024-03-16T10:11:24.744 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T10:11:24.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.860+0000 7fe6322db640 1 Processor -- start 2024-03-16T10:11:24.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.860+0000 7fe6322db640 1 -- start start 2024-03-16T10:11:24.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.860+0000 7fe6322db640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe62c066fc0 0x7fe62c0af980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.860+0000 7fe6322db640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe62c13e2f0 0x7fe62c14eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.860+0000 7fe6322db640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe62c0b0200 con 0x7fe62c0673a0 2024-03-16T10:11:24.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.860+0000 7fe6322db640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fe62c0b04e0 con 0x7fe62c066fc0 2024-03-16T10:11:24.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.860+0000 7fe6322db640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe62c0b5020 con 0x7fe62c13e2f0 2024-03-16T10:11:24.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.860+0000 7fe62b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe62c0673a0 0x7fe62c0b54b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46166/0 (socket says 172.21.15.17:46166) 2024-03-16T10:11:24.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.860+0000 7fe62b7fe640 1 -- 172.21.15.17:0/2808423996 learned_addr learned my addr 172.21.15.17:0/2808423996 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:24.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.860+0000 7fe630878640 1 --2- 172.21.15.17:0/2808423996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe62c13e2f0 0x7fe62c14eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.860+0000 7fe62bfff640 1 --2- 172.21.15.17:0/2808423996 >> v2:172.21.15.174:3300/0 conn(0x7fe62c066fc0 0x7fe62c0af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.860+0000 7fe62affd640 1 -- 172.21.15.17:0/2808423996 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1123872089 0 0) 0x7fe62c0b0200 con 0x7fe62c0673a0 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.860+0000 7fe62affd640 1 -- 172.21.15.17:0/2808423996 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe614003530 con 0x7fe62c0673a0 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 -- 172.21.15.17:0/2808423996 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1446609306 0 0) 0x7fe614003530 con 0x7fe62c0673a0 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 -- 172.21.15.17:0/2808423996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe62c13e2f0 msgr2=0x7fe62c14eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 --2- 172.21.15.17:0/2808423996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe62c13e2f0 0x7fe62c14eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 -- 172.21.15.17:0/2808423996 >> v2:172.21.15.174:3300/0 conn(0x7fe62c066fc0 msgr2=0x7fe62c0af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 --2- 172.21.15.17:0/2808423996 >> v2:172.21.15.174:3300/0 conn(0x7fe62c066fc0 0x7fe62c0af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 -- 172.21.15.17:0/2808423996 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c070720 con 0x7fe62c0673a0 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 -- 172.21.15.17:0/2808423996 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fe620001680 con 0x7fe62c0673a0 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 -- 172.21.15.17:0/2808423996 >> v1:172.21.15.17:6789/0 conn(0x7fe62c0673a0 legacy=0x7fe62c0b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 --2- 172.21.15.17:0/2808423996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe62c13e2f0 0x7fe6140039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 --2- 172.21.15.17:0/2808423996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe614003f40 0x7fe614004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 --2- 172.21.15.17:0/2808423996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe614004d50 0x7fe6140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 -- 172.21.15.17:0/2808423996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c0b04e0 con 0x7fe614003f40 2024-03-16T10:11:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 -- 172.21.15.17:0/2808423996 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c0b5020 con 0x7fe614004d50 2024-03-16T10:11:24.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 -- 172.21.15.17:0/2808423996 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe61400ba20 con 0x7fe62c13e2f0 2024-03-16T10:11:24.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62b7fe640 1 --2- 172.21.15.17:0/2808423996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe614003f40 0x7fe614004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe630878640 1 --2- 172.21.15.17:0/2808423996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe614004d50 0x7fe6140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62bfff640 1 --2- 172.21.15.17:0/2808423996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe62c13e2f0 0x7fe6140039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62b7fe640 1 -- 172.21.15.17:0/2808423996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe62c13e2f0 msgr2=0x7fe6140039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:24.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62b7fe640 1 --2- 172.21.15.17:0/2808423996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe62c13e2f0 0x7fe6140039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62b7fe640 1 -- 172.21.15.17:0/2808423996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe614004d50 msgr2=0x7fe6140051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:24.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62b7fe640 1 --2- 172.21.15.17:0/2808423996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe614004d50 0x7fe6140051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62b7fe640 1 -- 172.21.15.17:0/2808423996 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe614003530 con 0x7fe614003f40 2024-03-16T10:11:24.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62bfff640 1 --2- 172.21.15.17:0/2808423996 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe62c13e2f0 0x7fe6140039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:24.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62b7fe640 1 --2- 172.21.15.17:0/2808423996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe614003f40 0x7fe614004810 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe62000c210 tx=0x7fe62001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc58b01473c0c586 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:24.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 -- 172.21.15.17:0/2808423996 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe620007d80 con 0x7fe614003f40 2024-03-16T10:11:24.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 -- 172.21.15.17:0/2808423996 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6200a9020 con 0x7fe614003f40 2024-03-16T10:11:24.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe62affd640 1 -- 172.21.15.17:0/2808423996 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe620005ae0 con 0x7fe614003f40 2024-03-16T10:11:24.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe6322db640 1 -- 172.21.15.17:0/2808423996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe614003f40 msgr2=0x7fe614004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe6322db640 1 --2- 172.21.15.17:0/2808423996 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe614003f40 0x7fe614004810 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe62000c210 tx=0x7fe62001cca0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe630878640 1 -- 172.21.15.17:0/2808423996 reap_dead start 2024-03-16T10:11:24.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe6322db640 1 -- 172.21.15.17:0/2808423996 shutdown_connections 2024-03-16T10:11:24.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe6322db640 1 -- 172.21.15.17:0/2808423996 >> 172.21.15.17:0/2808423996 conn(0x7fe62c0b5e10 msgr2=0x7fe62c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:24.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe6322db640 1 -- 172.21.15.17:0/2808423996 shutdown_connections 2024-03-16T10:11:24.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.864+0000 7fe6322db640 1 -- 172.21.15.17:0/2808423996 wait complete. 2024-03-16T10:11:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe6322db640 1 Processor -- start 2024-03-16T10:11:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe6322db640 1 -- start start 2024-03-16T10:11:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe6322db640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe614003f40 0x7fe62c13c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe6322db640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe62c14aec0 0x7fe62c13b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe6322db640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe62c14b340 0x7fe62c13bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe6322db640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c14f340 con 0x7fe62c14b340 2024-03-16T10:11:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe6322db640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c14f4c0 con 0x7fe62c14aec0 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe6322db640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe62c14f640 con 0x7fe614003f40 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe630878640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe62c14b340 0x7fe62c13bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe630878640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe62c14b340 0x7fe62c13bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55442/0 (socket says 172.21.15.17:55442) 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe630878640 1 -- 172.21.15.17:0/3965612168 learned_addr learned my addr 172.21.15.17:0/3965612168 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe62b7fe640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe62c14aec0 0x7fe62c13b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe62bfff640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe614003f40 0x7fe62c13c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe630878640 1 -- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe614003f40 msgr2=0x7fe62c13c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe630878640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe614003f40 0x7fe62c13c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe630878640 1 -- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe62c14aec0 msgr2=0x7fe62c13b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe630878640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe62c14aec0 0x7fe62c13b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe630878640 1 -- 172.21.15.17:0/3965612168 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c14c620 con 0x7fe62c14b340 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe62b7fe640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe62c14aec0 0x7fe62c13b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe630878640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe62c14b340 0x7fe62c13bbc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe62c06fa20 tx=0x7fe6240115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe628ff9640 1 -- 172.21.15.17:0/3965612168 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe624011e10 con 0x7fe62c14b340 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe628ff9640 1 -- 172.21.15.17:0/3965612168 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe62401b070 con 0x7fe62c14b340 2024-03-16T10:11:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe6322db640 1 -- 172.21.15.17:0/3965612168 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62c149b40 con 0x7fe62c14b340 2024-03-16T10:11:24.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe628ff9640 1 -- 172.21.15.17:0/3965612168 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6240039a0 con 0x7fe62c14b340 2024-03-16T10:11:24.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe6322db640 1 -- 172.21.15.17:0/3965612168 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62c140090 con 0x7fe62c14b340 2024-03-16T10:11:24.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe6322db640 1 -- 172.21.15.17:0/3965612168 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe62c1444a0 con 0x7fe62c14b340 2024-03-16T10:11:24.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.868+0000 7fe628ff9640 1 -- 172.21.15.17:0/3965612168 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe624003b60 con 0x7fe62c14b340 2024-03-16T10:11:24.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.872+0000 7fe628ff9640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe618067690 0x7fe618069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:24.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.872+0000 7fe628ff9640 1 -- 172.21.15.17:0/3965612168 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe624086d30 con 0x7fe62c14b340 2024-03-16T10:11:24.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.872+0000 7fe62bfff640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe618067690 0x7fe618069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:24.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.872+0000 7fe62bfff640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe618067690 0x7fe618069b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009ac0 tx=0x7fe61c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:24.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:24.880+0000 7fe628ff9640 1 -- 172.21.15.17:0/3965612168 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe62c1444a0 con 0x7fe62c14b340 2024-03-16T10:11:25.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.052+0000 7fe6322db640 1 -- 172.21.15.17:0/3965612168 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe62c06a130 con 0x7fe62c14b340 2024-03-16T10:11:25.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.052+0000 7fe628ff9640 1 -- 172.21.15.17:0/3965612168 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fe624025020 con 0x7fe62c14b340 2024-03-16T10:11:25.057 INFO:teuthology.orchestra.run.smithi017.stderr:listed 0 entries 2024-03-16T10:11:25.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.052+0000 7fe6322db640 1 -- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe618067690 msgr2=0x7fe618069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:25.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.052+0000 7fe6322db640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe618067690 0x7fe618069b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009ac0 tx=0x7fe61c008040 comp rx=0 tx=0).stop 2024-03-16T10:11:25.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.052+0000 7fe6322db640 1 -- 172.21.15.17:0/3965612168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe62c14b340 msgr2=0x7fe62c13bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:25.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.052+0000 7fe6322db640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe62c14b340 0x7fe62c13bbc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe62c06fa20 tx=0x7fe6240115a0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.052+0000 7fe6322db640 1 -- 172.21.15.17:0/3965612168 shutdown_connections 2024-03-16T10:11:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.052+0000 7fe6322db640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe618067690 0x7fe618069b50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.052+0000 7fe6322db640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe62c14b340 0x7fe62c13bbc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.052+0000 7fe6322db640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe62c14aec0 0x7fe62c13b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.052+0000 7fe6322db640 1 --2- 172.21.15.17:0/3965612168 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe614003f40 0x7fe62c13c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.052+0000 7fe6322db640 1 -- 172.21.15.17:0/3965612168 >> 172.21.15.17:0/3965612168 conn(0x7fe62c0b5e10 msgr2=0x7fe62c06d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.052+0000 7fe6322db640 1 -- 172.21.15.17:0/3965612168 shutdown_connections 2024-03-16T10:11:25.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.052+0000 7fe6322db640 1 -- 172.21.15.17:0/3965612168 wait complete. 2024-03-16T10:11:25.075 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi017.front.sepia.ceph.com... 2024-03-16T10:11:25.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:25.075 DEBUG:teuthology.orchestra.run.smithi017:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:25.091 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:11:25.091 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi017.front.sepia.ceph.com 2024-03-16T10:11:25.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:25.091 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-16T10:11:25.150 INFO:tasks.cephfs.cephfs_test_case:[] 2024-03-16T10:11:25.150 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:11:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61ccebb640 1 Processor -- start 2024-03-16T10:11:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61ccebb640 1 -- start start 2024-03-16T10:11:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61ccebb640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f61c8067220 0x7f61c80b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61ccebb640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f61c80991a0 0x7f61c814b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61ccebb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c814b6a0 0x7f61c80b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61ccebb640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f61c80af590 con 0x7f61c80991a0 2024-03-16T10:11:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61ccebb640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f61c80ae840 con 0x7f61c8067220 2024-03-16T10:11:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61ccebb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f61c80b0620 con 0x7f61c814b6a0 2024-03-16T10:11:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61c5d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f61c80991a0 0x7f61c814b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61c5d9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f61c80991a0 0x7f61c814b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55448/0 (socket says 172.21.15.17:55448) 2024-03-16T10:11:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61c5d9b640 1 -- 172.21.15.17:0/3401996341 learned_addr learned my addr 172.21.15.17:0/3401996341 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61c659c640 1 --2- 172.21.15.17:0/3401996341 >> v2:172.21.15.174:3300/0 conn(0x7f61c8067220 0x7f61c80b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61c6d9d640 1 --2- 172.21.15.17:0/3401996341 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c814b6a0 0x7f61c80b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61c5d9b640 1 -- 172.21.15.17:0/3401996341 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c814b6a0 msgr2=0x7f61c80b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61c5d9b640 1 --2- 172.21.15.17:0/3401996341 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c814b6a0 0x7f61c80b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61c5d9b640 1 -- 172.21.15.17:0/3401996341 >> v2:172.21.15.174:3300/0 conn(0x7f61c8067220 msgr2=0x7f61c80b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61c5d9b640 1 --2- 172.21.15.17:0/3401996341 >> v2:172.21.15.174:3300/0 conn(0x7f61c8067220 0x7f61c80b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61c5d9b640 1 -- 172.21.15.17:0/3401996341 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c814bb10 con 0x7f61c80991a0 2024-03-16T10:11:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.320+0000 7f61c6d9d640 1 --2- 172.21.15.17:0/3401996341 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c814b6a0 0x7f61c80b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61c5d9b640 1 --2- 172.21.15.17:0/3401996341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c80991a0 0x7f61c814b160 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f61c8099070 tx=0x7f61bc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c79050c88b92ac server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61c559a640 1 -- 172.21.15.17:0/3401996341 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0b0070 con 0x7f61c80991a0 2024-03-16T10:11:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61c559a640 1 -- 172.21.15.17:0/3401996341 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61bc0ab600 con 0x7f61c80991a0 2024-03-16T10:11:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61c559a640 1 -- 172.21.15.17:0/3401996341 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0ab9d0 con 0x7f61c80991a0 2024-03-16T10:11:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61ccebb640 1 -- 172.21.15.17:0/3401996341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c80991a0 msgr2=0x7f61c814b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61ccebb640 1 --2- 172.21.15.17:0/3401996341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c80991a0 0x7f61c814b160 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f61c8099070 tx=0x7f61bc01c490 comp rx=0 tx=0).stop 2024-03-16T10:11:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61ccebb640 1 -- 172.21.15.17:0/3401996341 shutdown_connections 2024-03-16T10:11:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61ccebb640 1 --2- 172.21.15.17:0/3401996341 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c814b6a0 0x7f61c80b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61ccebb640 1 --2- 172.21.15.17:0/3401996341 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c80991a0 0x7f61c814b160 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61ccebb640 1 --2- 172.21.15.17:0/3401996341 >> v2:172.21.15.174:3300/0 conn(0x7f61c8067220 0x7f61c80b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61ccebb640 1 -- 172.21.15.17:0/3401996341 >> 172.21.15.17:0/3401996341 conn(0x7f61c80b5f10 msgr2=0x7f61c80b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61ccebb640 1 -- 172.21.15.17:0/3401996341 shutdown_connections 2024-03-16T10:11:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61ccebb640 1 -- 172.21.15.17:0/3401996341 wait complete. 2024-03-16T10:11:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61ccebb640 1 Processor -- start 2024-03-16T10:11:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.324+0000 7f61ccebb640 1 -- start start 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61ccebb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c8067220 0x7f61c813da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61ccebb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c80991a0 0x7f61c81e4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61ccebb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c814b6a0 0x7f61c81e0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61ccebb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f61c80b5b80 con 0x7f61c814b6a0 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61ccebb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61c80b5850 con 0x7f61c80991a0 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61ccebb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f61c80b5490 con 0x7f61c8067220 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61c6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c814b6a0 0x7f61c81e0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61c6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c814b6a0 0x7f61c81e0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55456/0 (socket says 172.21.15.17:55456) 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61c6d9d640 1 -- 172.21.15.17:0/3895529321 learned_addr learned my addr 172.21.15.17:0/3895529321 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61c5d9b640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c80991a0 0x7f61c81e4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61c659c640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c8067220 0x7f61c813da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61c6d9d640 1 -- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c8067220 msgr2=0x7f61c813da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61c6d9d640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c8067220 0x7f61c813da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61c6d9d640 1 -- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c80991a0 msgr2=0x7f61c81e4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61c6d9d640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c80991a0 0x7f61c81e4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61c6d9d640 1 -- 172.21.15.17:0/3895529321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c81e36a0 con 0x7f61c814b6a0 2024-03-16T10:11:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61c659c640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c8067220 0x7f61c813da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61c6d9d640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c814b6a0 0x7f61c81e0130 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f61c000cd80 tx=0x7f61c000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61b6ffd640 1 -- 172.21.15.17:0/3895529321 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c000f680 con 0x7f61c814b6a0 2024-03-16T10:11:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61b6ffd640 1 -- 172.21.15.17:0/3895529321 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61c000f840 con 0x7f61c814b6a0 2024-03-16T10:11:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61b6ffd640 1 -- 172.21.15.17:0/3895529321 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c000fc30 con 0x7f61c814b6a0 2024-03-16T10:11:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61ccebb640 1 -- 172.21.15.17:0/3895529321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c81e3980 con 0x7f61c814b6a0 2024-03-16T10:11:25.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61ccebb640 1 -- 172.21.15.17:0/3895529321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c809ef60 con 0x7f61c814b6a0 2024-03-16T10:11:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61ccebb640 1 -- 172.21.15.17:0/3895529321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c80984d0 con 0x7f61c814b6a0 2024-03-16T10:11:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61b6ffd640 1 -- 172.21.15.17:0/3895529321 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f61c000e3e0 con 0x7f61c814b6a0 2024-03-16T10:11:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61b6ffd640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f61a8067660 0x7f61a8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61b6ffd640 1 -- 172.21.15.17:0/3895529321 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f61c0086630 con 0x7f61c814b6a0 2024-03-16T10:11:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.328+0000 7f61c659c640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f61a8067660 0x7f61a8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:25.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.332+0000 7f61c659c640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f61a8067660 0x7f61a8069b20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f61b80097c0 tx=0x7f61b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.332+0000 7f61b6ffd640 1 -- 172.21.15.17:0/3895529321 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f61c80984d0 con 0x7f61c814b6a0 2024-03-16T10:11:25.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.732+0000 7f61ccebb640 1 -- 172.21.15.17:0/3895529321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61c806a130 con 0x7f61c814b6a0 2024-03-16T10:11:25.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.732+0000 7f61b6ffd640 1 -- 172.21.15.17:0/3895529321 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7f61c0014030 con 0x7f61c814b6a0 2024-03-16T10:11:25.738 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:25.738 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1058712158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1058712158},{"type":"v1","addr":"172.21.15.174:6835","nonce":1058712158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:10:57.464714+0000","modified":"2024-03-16T10:11:00.509595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6833/3105182870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3105182870},{"type":"v1","addr":"172.21.15.17:6833","nonce":3105182870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-16T10:11:25.738 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 8 2024-03-16T10:11:25.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.736+0000 7f61ccebb640 1 -- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f61a8067660 msgr2=0x7f61a8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:25.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.736+0000 7f61ccebb640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f61a8067660 0x7f61a8069b20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f61b80097c0 tx=0x7f61b8008040 comp rx=0 tx=0).stop 2024-03-16T10:11:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.736+0000 7f61ccebb640 1 -- 172.21.15.17:0/3895529321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c814b6a0 msgr2=0x7f61c81e0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.736+0000 7f61ccebb640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c814b6a0 0x7f61c81e0130 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f61c000cd80 tx=0x7f61c000a650 comp rx=0 tx=0).stop 2024-03-16T10:11:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.736+0000 7f61ccebb640 1 -- 172.21.15.17:0/3895529321 shutdown_connections 2024-03-16T10:11:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.736+0000 7f61ccebb640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f61a8067660 0x7f61a8069b20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.736+0000 7f61ccebb640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61c814b6a0 0x7f61c81e0130 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.736+0000 7f61ccebb640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61c80991a0 0x7f61c81e4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.736+0000 7f61ccebb640 1 --2- 172.21.15.17:0/3895529321 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f61c8067220 0x7f61c813da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.736+0000 7f61ccebb640 1 -- 172.21.15.17:0/3895529321 >> 172.21.15.17:0/3895529321 conn(0x7f61c80b5f10 msgr2=0x7f61c8141be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.736+0000 7f61ccebb640 1 -- 172.21.15.17:0/3895529321 shutdown_connections 2024-03-16T10:11:25.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.736+0000 7f61ccebb640 1 -- 172.21.15.17:0/3895529321 wait complete. 2024-03-16T10:11:25.756 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:25.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcf519e640 1 Processor -- start 2024-03-16T10:11:25.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcf519e640 1 -- start start 2024-03-16T10:11:25.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcf519e640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdcf00b3c30 0x7fdcf0147970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:25.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcf519e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdcf0147eb0 0x7fdcf0151e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:25.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcf519e640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcf00b49b0 con 0x7fdcf0067220 2024-03-16T10:11:25.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcf519e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf00b4090 con 0x7fdcf0147eb0 2024-03-16T10:11:25.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcf519e640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fdcf0067600 con 0x7fdcf00b3c30 2024-03-16T10:11:25.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcef59e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdcf0147eb0 0x7fdcf0151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:25.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcef59e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdcf0147eb0 0x7fdcf0151e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55466/0 (socket says 172.21.15.17:55466) 2024-03-16T10:11:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcef59e640 1 -- 172.21.15.17:0/2532528310 learned_addr learned my addr 172.21.15.17:0/2532528310 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcee59c640 1 --2- 172.21.15.17:0/2532528310 >> v2:172.21.15.174:3300/0 conn(0x7fdcf00b3c30 0x7fdcf0147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcef59e640 1 -- 172.21.15.17:0/2532528310 >> v2:172.21.15.174:3300/0 conn(0x7fdcf00b3c30 msgr2=0x7fdcf0147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcef59e640 1 --2- 172.21.15.17:0/2532528310 >> v2:172.21.15.174:3300/0 conn(0x7fdcf00b3c30 0x7fdcf0147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcef59e640 1 -- 172.21.15.17:0/2532528310 >> v1:172.21.15.174:6789/0 conn(0x7fdcf0067220 legacy=0x7fdcf0144a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.884+0000 7fdcef59e640 1 -- 172.21.15.17:0/2532528310 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf006d170 con 0x7fdcf0147eb0 2024-03-16T10:11:25.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcef59e640 1 --2- 172.21.15.17:0/2532528310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcf0147eb0 0x7fdcf0151e50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00b9ee0 tx=0x7fdce801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e90528f503c78 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:25.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcedd9b640 1 -- 172.21.15.17:0/2532528310 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce80a5800 con 0x7fdcf0147eb0 2024-03-16T10:11:25.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcedd9b640 1 -- 172.21.15.17:0/2532528310 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce80a59a0 con 0x7fdcf0147eb0 2024-03-16T10:11:25.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcedd9b640 1 -- 172.21.15.17:0/2532528310 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce80a5d70 con 0x7fdcf0147eb0 2024-03-16T10:11:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 -- 172.21.15.17:0/2532528310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcf0147eb0 msgr2=0x7fdcf0151e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 --2- 172.21.15.17:0/2532528310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcf0147eb0 0x7fdcf0151e50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00b9ee0 tx=0x7fdce801c940 comp rx=0 tx=0).stop 2024-03-16T10:11:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 -- 172.21.15.17:0/2532528310 shutdown_connections 2024-03-16T10:11:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 --2- 172.21.15.17:0/2532528310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcf0147eb0 0x7fdcf0151e50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 --2- 172.21.15.17:0/2532528310 >> v2:172.21.15.174:3300/0 conn(0x7fdcf00b3c30 0x7fdcf0147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 -- 172.21.15.17:0/2532528310 >> 172.21.15.17:0/2532528310 conn(0x7fdcf006bc80 msgr2=0x7fdcf00b9230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 -- 172.21.15.17:0/2532528310 shutdown_connections 2024-03-16T10:11:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 -- 172.21.15.17:0/2532528310 wait complete. 2024-03-16T10:11:25.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 Processor -- start 2024-03-16T10:11:25.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 -- start start 2024-03-16T10:11:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcf0067220 0x7fdcf01413e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcf00b3c30 0x7fdcf0148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcf0147eb0 0x7fdcf00727a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0145110 con 0x7fdcf0147eb0 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf00b5a60 con 0x7fdcf00b3c30 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcf519e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf0141920 con 0x7fdcf0067220 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcef59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcf0147eb0 0x7fdcf00727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcef59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcf0147eb0 0x7fdcf00727a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55482/0 (socket says 172.21.15.17:55482) 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcef59e640 1 -- 172.21.15.17:0/668969423 learned_addr learned my addr 172.21.15.17:0/668969423 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcee59c640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcf00b3c30 0x7fdcf0148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdceed9d640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcf0067220 0x7fdcf01413e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcef59e640 1 -- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcf0067220 msgr2=0x7fdcf01413e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcef59e640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcf0067220 0x7fdcf01413e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcef59e640 1 -- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcf00b3c30 msgr2=0x7fdcf0148e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcef59e640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcf00b3c30 0x7fdcf0148e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcef59e640 1 -- 172.21.15.17:0/668969423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf014aa50 con 0x7fdcf0147eb0 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdceed9d640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcf0067220 0x7fdcf01413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.888+0000 7fdcef59e640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcf0147eb0 0x7fdcf00727a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdce8019550 tx=0x7fdce80196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.892+0000 7fdcd77fe640 1 -- 172.21.15.17:0/668969423 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce80bc020 con 0x7fdcf0147eb0 2024-03-16T10:11:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.892+0000 7fdcd77fe640 1 -- 172.21.15.17:0/668969423 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce80b8670 con 0x7fdcf0147eb0 2024-03-16T10:11:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.892+0000 7fdcd77fe640 1 -- 172.21.15.17:0/668969423 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce80b8a60 con 0x7fdcf0147eb0 2024-03-16T10:11:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.892+0000 7fdcf519e640 1 -- 172.21.15.17:0/668969423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf014ad30 con 0x7fdcf0147eb0 2024-03-16T10:11:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.892+0000 7fdcf519e640 1 -- 172.21.15.17:0/668969423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf014b4a0 con 0x7fdcf0147eb0 2024-03-16T10:11:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.892+0000 7fdcf519e640 1 -- 172.21.15.17:0/668969423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf006a130 con 0x7fdcf0147eb0 2024-03-16T10:11:25.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.892+0000 7fdcd77fe640 1 -- 172.21.15.17:0/668969423 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fdce80bf020 con 0x7fdcf0147eb0 2024-03-16T10:11:25.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.892+0000 7fdcd77fe640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fdcdc067660 0x7fdcdc069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:25.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.892+0000 7fdcd77fe640 1 -- 172.21.15.17:0/668969423 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdce8120950 con 0x7fdcf0147eb0 2024-03-16T10:11:25.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.892+0000 7fdceed9d640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fdcdc067660 0x7fdcdc069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:25.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.892+0000 7fdceed9d640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fdcdc067660 0x7fdcdc069b20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00afc30 tx=0x7fdce0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:25.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:25.896+0000 7fdcd77fe640 1 -- 172.21.15.17:0/668969423 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdce80c6030 con 0x7fdcf0147eb0 2024-03-16T10:11:26.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.056+0000 7fdcf519e640 1 -- 172.21.15.17:0/668969423 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdcf00af980 con 0x7fdcf0147eb0 2024-03-16T10:11:26.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.060+0000 7fdcd77fe640 1 -- 172.21.15.17:0/668969423 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7fdce80165c0 con 0x7fdcf0147eb0 2024-03-16T10:11:26.065 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:26.065 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:26.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.060+0000 7fdcf519e640 1 -- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fdcdc067660 msgr2=0x7fdcdc069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.060+0000 7fdcf519e640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fdcdc067660 0x7fdcdc069b20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00afc30 tx=0x7fdce0009040 comp rx=0 tx=0).stop 2024-03-16T10:11:26.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.064+0000 7fdcf519e640 1 -- 172.21.15.17:0/668969423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcf0147eb0 msgr2=0x7fdcf00727a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.064+0000 7fdcf519e640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcf0147eb0 0x7fdcf00727a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdce8019550 tx=0x7fdce80196e0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.064+0000 7fdcf519e640 1 -- 172.21.15.17:0/668969423 shutdown_connections 2024-03-16T10:11:26.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.064+0000 7fdcf519e640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fdcdc067660 0x7fdcdc069b20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.064+0000 7fdcf519e640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdcf0147eb0 0x7fdcf00727a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.064+0000 7fdcf519e640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcf00b3c30 0x7fdcf0148e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.064+0000 7fdcf519e640 1 --2- 172.21.15.17:0/668969423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdcf0067220 0x7fdcf01413e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.064+0000 7fdcf519e640 1 -- 172.21.15.17:0/668969423 >> 172.21.15.17:0/668969423 conn(0x7fdcf006bc80 msgr2=0x7fdcf0145e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:26.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.064+0000 7fdcf519e640 1 -- 172.21.15.17:0/668969423 shutdown_connections 2024-03-16T10:11:26.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.064+0000 7fdcf519e640 1 -- 172.21.15.17:0/668969423 wait complete. 2024-03-16T10:11:26.085 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:10:55.723734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:10:56.459894+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:11:26.085 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:11:26.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e2fea640 1 Processor -- start 2024-03-16T10:11:26.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e2fea640 1 -- start start 2024-03-16T10:11:26.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e2fea640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10dc0b7770 0x7f10dc145200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e2fea640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f10dc073070 0x7f10dc0b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e2fea640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10dc0b6fd0 con 0x7f10dc145740 2024-03-16T10:11:26.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e2fea640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f10dc0b64f0 con 0x7f10dc0b7770 2024-03-16T10:11:26.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e2fea640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f10dc0b67d0 con 0x7f10dc073070 2024-03-16T10:11:26.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e0d86640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10dc0b7770 0x7f10dc145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e0d86640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10dc0b7770 0x7f10dc145200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55484/0 (socket says 172.21.15.17:55484) 2024-03-16T10:11:26.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e1587640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f10dc145740 0x7f10dc148770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46174/0 (socket says 172.21.15.17:46174) 2024-03-16T10:11:26.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e0d86640 1 -- 172.21.15.17:0/2929496109 learned_addr learned my addr 172.21.15.17:0/2929496109 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:26.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10dbfff640 1 --2- 172.21.15.17:0/2929496109 >> v2:172.21.15.174:3300/0 conn(0x7f10dc073070 0x7f10dc0b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e0d86640 1 -- 172.21.15.17:0/2929496109 >> v2:172.21.15.174:3300/0 conn(0x7f10dc073070 msgr2=0x7f10dc0b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:26.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e0d86640 1 --2- 172.21.15.17:0/2929496109 >> v2:172.21.15.174:3300/0 conn(0x7f10dc073070 0x7f10dc0b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e0d86640 1 -- 172.21.15.17:0/2929496109 >> v1:172.21.15.17:6789/0 conn(0x7f10dc145740 legacy=0x7f10dc148770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10db7fe640 1 -- 172.21.15.17:0/2929496109 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3366685139 0 0) 0x7f10dc0b6fd0 con 0x7f10dc145740 2024-03-16T10:11:26.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10e0d86640 1 -- 172.21.15.17:0/2929496109 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc0b6ab0 con 0x7f10dc0b7770 2024-03-16T10:11:26.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.212+0000 7f10dbfff640 1 --2- 172.21.15.17:0/2929496109 >> v2:172.21.15.174:3300/0 conn(0x7f10dc073070 0x7f10dc0b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:26.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.216+0000 7f10e0d86640 1 --2- 172.21.15.17:0/2929496109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10dc0b7770 0x7f10dc145200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0b8140 tx=0x7f10d001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2019337a2f0d06 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:26.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.216+0000 7f10db7fe640 1 -- 172.21.15.17:0/2929496109 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d00b1070 con 0x7f10dc0b7770 2024-03-16T10:11:26.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.216+0000 7f10db7fe640 1 -- 172.21.15.17:0/2929496109 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d00ac640 con 0x7f10dc0b7770 2024-03-16T10:11:26.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.216+0000 7f10db7fe640 1 -- 172.21.15.17:0/2929496109 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d00aca10 con 0x7f10dc0b7770 2024-03-16T10:11:26.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.216+0000 7f10e2fea640 1 -- 172.21.15.17:0/2929496109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10dc0b7770 msgr2=0x7f10dc145200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.216+0000 7f10e2fea640 1 --2- 172.21.15.17:0/2929496109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10dc0b7770 0x7f10dc145200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f10dc0b8140 tx=0x7f10d001c400 comp rx=0 tx=0).stop 2024-03-16T10:11:26.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.216+0000 7f10e2fea640 1 -- 172.21.15.17:0/2929496109 shutdown_connections 2024-03-16T10:11:26.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.216+0000 7f10e2fea640 1 --2- 172.21.15.17:0/2929496109 >> v2:172.21.15.174:3300/0 conn(0x7f10dc073070 0x7f10dc0b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.216+0000 7f10e2fea640 1 --2- 172.21.15.17:0/2929496109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10dc0b7770 0x7f10dc145200 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.216+0000 7f10e2fea640 1 -- 172.21.15.17:0/2929496109 >> 172.21.15.17:0/2929496109 conn(0x7f10dc06b290 msgr2=0x7f10dc06b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:26.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.216+0000 7f10e2fea640 1 -- 172.21.15.17:0/2929496109 shutdown_connections 2024-03-16T10:11:26.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.216+0000 7f10e2fea640 1 -- 172.21.15.17:0/2929496109 wait complete. 2024-03-16T10:11:26.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e2fea640 1 Processor -- start 2024-03-16T10:11:26.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e2fea640 1 -- start start 2024-03-16T10:11:26.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e2fea640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10dc073070 0x7f10dc1e3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e2fea640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10dc0b7770 0x7f10dc1e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e2fea640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10dc145740 0x7f10dc132900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e2fea640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc152c40 con 0x7f10dc145740 2024-03-16T10:11:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e2fea640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc152ac0 con 0x7f10dc073070 2024-03-16T10:11:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e2fea640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f10dc152dc0 con 0x7f10dc0b7770 2024-03-16T10:11:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e1587640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10dc145740 0x7f10dc132900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e1587640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10dc145740 0x7f10dc132900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55500/0 (socket says 172.21.15.17:55500) 2024-03-16T10:11:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e1587640 1 -- 172.21.15.17:0/2620694609 learned_addr learned my addr 172.21.15.17:0/2620694609 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e0d86640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10dc073070 0x7f10dc1e3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10dbfff640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10dc0b7770 0x7f10dc1e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e1587640 1 -- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10dc0b7770 msgr2=0x7f10dc1e40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e1587640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10dc0b7770 0x7f10dc1e40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e1587640 1 -- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10dc073070 msgr2=0x7f10dc1e3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e1587640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10dc073070 0x7f10dc1e3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e1587640 1 -- 172.21.15.17:0/2620694609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc1377d0 con 0x7f10dc145740 2024-03-16T10:11:26.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e0d86640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10dc073070 0x7f10dc1e3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:26.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e1587640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10dc145740 0x7f10dc132900 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f10d400c6d0 tx=0x7f10d400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:26.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10d97fa640 1 -- 172.21.15.17:0/2620694609 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4018070 con 0x7f10dc145740 2024-03-16T10:11:26.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10d97fa640 1 -- 172.21.15.17:0/2620694609 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10d4004d70 con 0x7f10dc145740 2024-03-16T10:11:26.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e2fea640 1 -- 172.21.15.17:0/2620694609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc138870 con 0x7f10dc145740 2024-03-16T10:11:26.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10d97fa640 1 -- 172.21.15.17:0/2620694609 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d400a880 con 0x7f10dc145740 2024-03-16T10:11:26.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e2fea640 1 -- 172.21.15.17:0/2620694609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc1372b0 con 0x7f10dc145740 2024-03-16T10:11:26.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e2fea640 1 -- 172.21.15.17:0/2620694609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10dc1374b0 con 0x7f10dc145740 2024-03-16T10:11:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10d97fa640 1 -- 172.21.15.17:0/2620694609 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f10d401b070 con 0x7f10dc145740 2024-03-16T10:11:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10d97fa640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f10c0067660 0x7f10c0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10d97fa640 1 -- 172.21.15.17:0/2620694609 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f10d4084db0 con 0x7f10dc145740 2024-03-16T10:11:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.220+0000 7f10e0d86640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f10c0067660 0x7f10c0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.224+0000 7f10e0d86640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f10c0067660 0x7f10c0069b20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f10d00ae000 tx=0x7f10d00a6070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:26.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.224+0000 7f10d97fa640 1 -- 172.21.15.17:0/2620694609 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f10dc1374b0 con 0x7f10dc145740 2024-03-16T10:11:26.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.388+0000 7f10e2fea640 1 -- 172.21.15.17:0/2620694609 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f10dc142c10 con 0x7f10dc145740 2024-03-16T10:11:26.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.392+0000 7f10d97fa640 1 -- 172.21.15.17:0/2620694609 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f10d4055a20 con 0x7f10dc145740 2024-03-16T10:11:26.396 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:11:26.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.392+0000 7f10e2fea640 1 -- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f10c0067660 msgr2=0x7f10c0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.392+0000 7f10e2fea640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f10c0067660 0x7f10c0069b20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f10d00ae000 tx=0x7f10d00a6070 comp rx=0 tx=0).stop 2024-03-16T10:11:26.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.392+0000 7f10e2fea640 1 -- 172.21.15.17:0/2620694609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10dc145740 msgr2=0x7f10dc132900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.392+0000 7f10e2fea640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10dc145740 0x7f10dc132900 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f10d400c6d0 tx=0x7f10d400cb90 comp rx=0 tx=0).stop 2024-03-16T10:11:26.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.392+0000 7f10e2fea640 1 -- 172.21.15.17:0/2620694609 shutdown_connections 2024-03-16T10:11:26.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.392+0000 7f10e2fea640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f10c0067660 0x7f10c0069b20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.392+0000 7f10e2fea640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10dc145740 0x7f10dc132900 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.392+0000 7f10e2fea640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10dc0b7770 0x7f10dc1e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.392+0000 7f10e2fea640 1 --2- 172.21.15.17:0/2620694609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10dc073070 0x7f10dc1e3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.392+0000 7f10e2fea640 1 -- 172.21.15.17:0/2620694609 >> 172.21.15.17:0/2620694609 conn(0x7f10dc06b290 msgr2=0x7f10dc0b95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:26.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.392+0000 7f10e2fea640 1 -- 172.21.15.17:0/2620694609 shutdown_connections 2024-03-16T10:11:26.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.392+0000 7f10e2fea640 1 -- 172.21.15.17:0/2620694609 wait complete. 2024-03-16T10:11:26.414 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:11:26.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.532+0000 7efdfcdfc640 1 Processor -- start 2024-03-16T10:11:26.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdfcdfc640 1 -- start start 2024-03-16T10:11:26.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdfcdfc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf80b5740 0x7efdf80aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdfcdfc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdf80b5b20 con 0x7efdf814c9a0 2024-03-16T10:11:26.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdfcdfc640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdf80af720 con 0x7efdf806ede0 2024-03-16T10:11:26.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdfcdfc640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7efdf806e950 con 0x7efdf80b5740 2024-03-16T10:11:26.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf6d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7efdf814c9a0 0x7efdf814ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46182/0 (socket says 172.21.15.17:46182) 2024-03-16T10:11:26.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf6d9d640 1 -- 172.21.15.17:0/2940632292 learned_addr learned my addr 172.21.15.17:0/2940632292 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:26.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 -- 172.21.15.17:0/2940632292 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2235799965 0 0) 0x7efdf80b5b20 con 0x7efdf814c9a0 2024-03-16T10:11:26.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 -- 172.21.15.17:0/2940632292 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efde4003530 con 0x7efdf814c9a0 2024-03-16T10:11:26.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf659c640 1 --2- 172.21.15.17:0/2940632292 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf80b5740 0x7efdf80aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 -- 172.21.15.17:0/2940632292 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2346173619 0 0) 0x7efde4003530 con 0x7efdf814c9a0 2024-03-16T10:11:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 -- 172.21.15.17:0/2940632292 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf80b5740 msgr2=0x7efdf80aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 --2- 172.21.15.17:0/2940632292 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf80b5740 0x7efdf80aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 -- 172.21.15.17:0/2940632292 >> v1:172.21.15.174:6789/0 conn(0x7efdf806ede0 legacy=0x7efdf8071ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 -- 172.21.15.17:0/2940632292 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf80afa00 con 0x7efdf814c9a0 2024-03-16T10:11:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 -- 172.21.15.17:0/2940632292 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7efdf0001a00 con 0x7efdf814c9a0 2024-03-16T10:11:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 -- 172.21.15.17:0/2940632292 >> v1:172.21.15.17:6789/0 conn(0x7efdf814c9a0 legacy=0x7efdf814ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 --2- 172.21.15.17:0/2940632292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efde40039f0 0x7efde4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 --2- 172.21.15.17:0/2940632292 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf80b5740 0x7efde4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 --2- 172.21.15.17:0/2940632292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde4004d50 0x7efde40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 -- 172.21.15.17:0/2940632292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efdf806e950 con 0x7efde4004d50 2024-03-16T10:11:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 -- 172.21.15.17:0/2940632292 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efde400bab0 con 0x7efde40039f0 2024-03-16T10:11:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf5d9b640 1 -- 172.21.15.17:0/2940632292 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7efde400dec0 con 0x7efdf80b5740 2024-03-16T10:11:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf6d9d640 1 --2- 172.21.15.17:0/2940632292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde4004d50 0x7efde40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf659c640 1 --2- 172.21.15.17:0/2940632292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efde40039f0 0x7efde4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdeffff640 1 --2- 172.21.15.17:0/2940632292 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf80b5740 0x7efde4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf6d9d640 1 -- 172.21.15.17:0/2940632292 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf80b5740 msgr2=0x7efde4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:26.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf6d9d640 1 --2- 172.21.15.17:0/2940632292 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf80b5740 0x7efde4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf6d9d640 1 -- 172.21.15.17:0/2940632292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efde40039f0 msgr2=0x7efde4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:26.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf6d9d640 1 --2- 172.21.15.17:0/2940632292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efde40039f0 0x7efde4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.536+0000 7efdf6d9d640 1 -- 172.21.15.17:0/2940632292 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efde4003530 con 0x7efde4004d50 2024-03-16T10:11:26.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdf6d9d640 1 --2- 172.21.15.17:0/2940632292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde4004d50 0x7efde40051d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efdf00013e0 tx=0x7efdf001d200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de94d2147e932fd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:26.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdf5d9b640 1 -- 172.21.15.17:0/2940632292 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf000a3b0 con 0x7efde4004d50 2024-03-16T10:11:26.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdf5d9b640 1 -- 172.21.15.17:0/2940632292 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf00ad650 con 0x7efde4004d50 2024-03-16T10:11:26.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdf5d9b640 1 -- 172.21.15.17:0/2940632292 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf00ada40 con 0x7efde4004d50 2024-03-16T10:11:26.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2940632292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde4004d50 msgr2=0x7efde40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdfcdfc640 1 --2- 172.21.15.17:0/2940632292 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efde4004d50 0x7efde40051d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efdf00013e0 tx=0x7efdf001d200 comp rx=0 tx=0).stop 2024-03-16T10:11:26.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdf6d9d640 1 -- 172.21.15.17:0/2940632292 reap_dead start 2024-03-16T10:11:26.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2940632292 shutdown_connections 2024-03-16T10:11:26.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2940632292 >> 172.21.15.17:0/2940632292 conn(0x7efdf80b65d0 msgr2=0x7efdf80b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:26.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2940632292 shutdown_connections 2024-03-16T10:11:26.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2940632292 wait complete. 2024-03-16T10:11:26.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdfcdfc640 1 Processor -- start 2024-03-16T10:11:26.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdfcdfc640 1 -- start start 2024-03-16T10:11:26.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdfcdfc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efde4004d50 0x7efdf813d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdfcdfc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdf80b1980 0x7efdf80b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdfcdfc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf813ac20 0x7efdf813b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdfcdfc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efdf80af4e0 con 0x7efdf80b1980 2024-03-16T10:11:26.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdfcdfc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efdf80af360 con 0x7efde4004d50 2024-03-16T10:11:26.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdfcdfc640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7efdf806f220 con 0x7efdf813ac20 2024-03-16T10:11:26.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdeffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdf80b1980 0x7efdf80b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdeffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdf80b1980 0x7efdf80b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55522/0 (socket says 172.21.15.17:55522) 2024-03-16T10:11:26.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.540+0000 7efdeffff640 1 -- 172.21.15.17:0/2500834433 learned_addr learned my addr 172.21.15.17:0/2500834433 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:26.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdf6d9d640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf813ac20 0x7efdf813b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdf659c640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efde4004d50 0x7efdf813d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdeffff640 1 -- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf813ac20 msgr2=0x7efdf813b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdeffff640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf813ac20 0x7efdf813b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdeffff640 1 -- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efde4004d50 msgr2=0x7efdf813d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdeffff640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efde4004d50 0x7efdf813d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdeffff640 1 -- 172.21.15.17:0/2500834433 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf80b3ae0 con 0x7efdf80b1980 2024-03-16T10:11:26.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdf659c640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efde4004d50 0x7efdf813d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:26.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdeffff640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdf80b1980 0x7efdf80b1de0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efde000e470 tx=0x7efde000e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:26.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdeeffd640 1 -- 172.21.15.17:0/2500834433 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde00052e0 con 0x7efdf80b1980 2024-03-16T10:11:26.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdeeffd640 1 -- 172.21.15.17:0/2500834433 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efde0018070 con 0x7efdf80b1980 2024-03-16T10:11:26.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2500834433 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf80b0ec0 con 0x7efdf80b1980 2024-03-16T10:11:26.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdeeffd640 1 -- 172.21.15.17:0/2500834433 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0005480 con 0x7efdf80b1980 2024-03-16T10:11:26.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2500834433 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf81444f0 con 0x7efdf80b1980 2024-03-16T10:11:26.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2500834433 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf806a130 con 0x7efdf80b1980 2024-03-16T10:11:26.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdeeffd640 1 -- 172.21.15.17:0/2500834433 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7efde0003840 con 0x7efdf80b1980 2024-03-16T10:11:26.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdeeffd640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efdd8067660 0x7efdd8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdeeffd640 1 -- 172.21.15.17:0/2500834433 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efde00854b0 con 0x7efdf80b1980 2024-03-16T10:11:26.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdf659c640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efdd8067660 0x7efdd8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.544+0000 7efdf659c640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efdd8067660 0x7efdd8069b20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efde8009a20 tx=0x7efde8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:26.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.548+0000 7efdeeffd640 1 -- 172.21.15.17:0/2500834433 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7efde00561c0 con 0x7efdf80b1980 2024-03-16T10:11:26.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.768+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2500834433 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efdf80b91c0 con 0x7efdf80b1980 2024-03-16T10:11:26.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.768+0000 7efdeeffd640 1 -- 172.21.15.17:0/2500834433 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7efde0055ac0 con 0x7efdf80b1980 2024-03-16T10:11:26.772 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-16T10:11:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.768+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efdd8067660 msgr2=0x7efdd8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.768+0000 7efdfcdfc640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efdd8067660 0x7efdd8069b20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efde8009a20 tx=0x7efde8008040 comp rx=0 tx=0).stop 2024-03-16T10:11:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.768+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2500834433 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdf80b1980 msgr2=0x7efdf80b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.768+0000 7efdfcdfc640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdf80b1980 0x7efdf80b1de0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efde000e470 tx=0x7efde000e930 comp rx=0 tx=0).stop 2024-03-16T10:11:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.768+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2500834433 shutdown_connections 2024-03-16T10:11:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.768+0000 7efdfcdfc640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7efdd8067660 0x7efdd8069b20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.768+0000 7efdfcdfc640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7efdf813ac20 0x7efdf813b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.768+0000 7efdfcdfc640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efdf80b1980 0x7efdf80b1de0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.768+0000 7efdfcdfc640 1 --2- 172.21.15.17:0/2500834433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efde4004d50 0x7efdf813d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.768+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2500834433 >> 172.21.15.17:0/2500834433 conn(0x7efdf80b65d0 msgr2=0x7efdf806b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:26.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.768+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2500834433 shutdown_connections 2024-03-16T10:11:26.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.768+0000 7efdfcdfc640 1 -- 172.21.15.17:0/2500834433 wait complete. 2024-03-16T10:11:26.791 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:11:26.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.908+0000 7f2ba2993640 1 Processor -- start 2024-03-16T10:11:26.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.908+0000 7f2ba2993640 1 -- start start 2024-03-16T10:11:26.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2ba2993640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b9c072b10 0x7f2b9c148740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2ba2993640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2b9c06f870 0x7f2b9c1422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2ba2993640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b9c072ef0 con 0x7f2b9c06fcd0 2024-03-16T10:11:26.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2ba2993640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f2b9c0b0360 con 0x7f2b9c06f870 2024-03-16T10:11:26.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2ba2993640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2b9c0b0c00 con 0x7f2b9c072b10 2024-03-16T10:11:26.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2b9b7fe640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2b9c06f870 0x7f2b9c1422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2b9bfff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b9c072b10 0x7f2b9c148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2b9b7fe640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2b9c06f870 0x7f2b9c1422b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.17:34164/0 (socket says 172.21.15.17:34164) 2024-03-16T10:11:26.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2ba0f30640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f2b9c06fcd0 0x7f2b9c0b56f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.17:59228/0 (socket says 172.21.15.17:59228) 2024-03-16T10:11:26.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2b9b7fe640 1 -- 172.21.15.17:0/3947060553 learned_addr learned my addr 172.21.15.17:0/3947060553 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:26.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2b9b7fe640 1 -- 172.21.15.17:0/3947060553 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b9c072b10 msgr2=0x7f2b9c148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:26.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2b9b7fe640 1 --2- 172.21.15.17:0/3947060553 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b9c072b10 0x7f2b9c148740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2b9b7fe640 1 -- 172.21.15.17:0/3947060553 >> v1:172.21.15.174:6789/0 conn(0x7f2b9c06fcd0 legacy=0x7f2b9c0b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2b9b7fe640 1 -- 172.21.15.17:0/3947060553 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9c0b0ee0 con 0x7f2b9c06f870 2024-03-16T10:11:26.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.912+0000 7f2b9bfff640 1 --2- 172.21.15.17:0/3947060553 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b9c072b10 0x7f2b9c148740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:11:26.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.916+0000 7f2b9b7fe640 1 --2- 172.21.15.17:0/3947060553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b9c06f870 0x7f2b9c1422b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009870 tx=0x7f2b9001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9a06d67c05ea3a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:26.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.916+0000 7f2b9affd640 1 -- 172.21.15.17:0/3947060553 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b900b0070 con 0x7f2b9c06f870 2024-03-16T10:11:26.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.916+0000 7f2b9affd640 1 -- 172.21.15.17:0/3947060553 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b900ab600 con 0x7f2b9c06f870 2024-03-16T10:11:26.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.916+0000 7f2b9affd640 1 -- 172.21.15.17:0/3947060553 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b900ab9d0 con 0x7f2b9c06f870 2024-03-16T10:11:26.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.916+0000 7f2ba2993640 1 -- 172.21.15.17:0/3947060553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b9c06f870 msgr2=0x7f2b9c1422b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.916+0000 7f2ba2993640 1 --2- 172.21.15.17:0/3947060553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b9c06f870 0x7f2b9c1422b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2b90009870 tx=0x7f2b9001c4d0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.916+0000 7f2ba2993640 1 -- 172.21.15.17:0/3947060553 shutdown_connections 2024-03-16T10:11:26.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.916+0000 7f2ba2993640 1 --2- 172.21.15.17:0/3947060553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b9c06f870 0x7f2b9c1422b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.916+0000 7f2ba2993640 1 --2- 172.21.15.17:0/3947060553 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b9c072b10 0x7f2b9c148740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.916+0000 7f2ba2993640 1 -- 172.21.15.17:0/3947060553 >> 172.21.15.17:0/3947060553 conn(0x7f2b9c0b77f0 msgr2=0x7f2b9c0b7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:26.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.916+0000 7f2ba2993640 1 -- 172.21.15.17:0/3947060553 shutdown_connections 2024-03-16T10:11:26.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.916+0000 7f2ba2993640 1 -- 172.21.15.17:0/3947060553 wait complete. 2024-03-16T10:11:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.916+0000 7f2ba2993640 1 Processor -- start 2024-03-16T10:11:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.916+0000 7f2ba2993640 1 -- start start 2024-03-16T10:11:26.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2ba2993640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b9c06f870 0x7f2b9c13d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2ba2993640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b9c06fcd0 0x7f2b9c1ddcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2ba2993640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b9c072b10 0x7f2b9c1de210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2ba2993640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c148e00 con 0x7f2b9c06fcd0 2024-03-16T10:11:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2ba2993640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b9c148c80 con 0x7f2b9c06f870 2024-03-16T10:11:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2ba2993640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2b9c152ab0 con 0x7f2b9c072b10 2024-03-16T10:11:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2b9b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b9c06fcd0 0x7f2b9c1ddcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2b9b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b9c06fcd0 0x7f2b9c1ddcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55536/0 (socket says 172.21.15.17:55536) 2024-03-16T10:11:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2b9b7fe640 1 -- 172.21.15.17:0/1205194632 learned_addr learned my addr 172.21.15.17:0/1205194632 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2b9bfff640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b9c06f870 0x7f2b9c13d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2ba0f30640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b9c072b10 0x7f2b9c1de210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2b9b7fe640 1 -- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b9c072b10 msgr2=0x7f2b9c1de210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2b9b7fe640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b9c072b10 0x7f2b9c1de210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2b9b7fe640 1 -- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b9c06f870 msgr2=0x7f2b9c13d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:26.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2b9b7fe640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b9c06f870 0x7f2b9c13d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:26.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2b9b7fe640 1 -- 172.21.15.17:0/1205194632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9c13a860 con 0x7f2b9c06fcd0 2024-03-16T10:11:26.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2ba0f30640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b9c072b10 0x7f2b9c1de210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:26.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2b9b7fe640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b9c06fcd0 0x7f2b9c1ddcd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2b900ae040 tx=0x7f2b90019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:26.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2b98ff9640 1 -- 172.21.15.17:0/1205194632 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b900b0040 con 0x7f2b9c06fcd0 2024-03-16T10:11:26.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2b98ff9640 1 -- 172.21.15.17:0/1205194632 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b900ab400 con 0x7f2b9c06fcd0 2024-03-16T10:11:26.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2ba2993640 1 -- 172.21.15.17:0/1205194632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9c13ab40 con 0x7f2b9c06fcd0 2024-03-16T10:11:26.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2b98ff9640 1 -- 172.21.15.17:0/1205194632 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b900b4670 con 0x7f2b9c06fcd0 2024-03-16T10:11:26.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2ba2993640 1 -- 172.21.15.17:0/1205194632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b9c13e220 con 0x7f2b9c06fcd0 2024-03-16T10:11:26.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2ba2993640 1 -- 172.21.15.17:0/1205194632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b9c06a130 con 0x7f2b9c06fcd0 2024-03-16T10:11:26.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.920+0000 7f2b98ff9640 1 -- 172.21.15.17:0/1205194632 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f2b900bb030 con 0x7f2b9c06fcd0 2024-03-16T10:11:26.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.924+0000 7f2b98ff9640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2b88067660 0x7f2b88069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:26.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.924+0000 7f2b98ff9640 1 -- 172.21.15.17:0/1205194632 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2b9011f690 con 0x7f2b9c06fcd0 2024-03-16T10:11:26.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.924+0000 7f2b9bfff640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2b88067660 0x7f2b88069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:26.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.924+0000 7f2b9bfff640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2b88067660 0x7f2b88069b20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009a50 tx=0x7f2b8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:26.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:26.924+0000 7f2b98ff9640 1 -- 172.21.15.17:0/1205194632 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2b900c2020 con 0x7f2b9c06fcd0 2024-03-16T10:11:27.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.152+0000 7f2ba2993640 1 -- 172.21.15.17:0/1205194632 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2b9c067a50 con 0x7f2b9c06fcd0 2024-03-16T10:11:27.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.152+0000 7f2b98ff9640 1 -- 172.21.15.17:0/1205194632 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f2b90016150 con 0x7f2b9c06fcd0 2024-03-16T10:11:27.158 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-16T10:11:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.156+0000 7f2ba2993640 1 -- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2b88067660 msgr2=0x7f2b88069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.156+0000 7f2ba2993640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2b88067660 0x7f2b88069b20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c009a50 tx=0x7f2b8c008040 comp rx=0 tx=0).stop 2024-03-16T10:11:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.156+0000 7f2ba2993640 1 -- 172.21.15.17:0/1205194632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b9c06fcd0 msgr2=0x7f2b9c1ddcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.156+0000 7f2ba2993640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b9c06fcd0 0x7f2b9c1ddcd0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2b900ae040 tx=0x7f2b90019270 comp rx=0 tx=0).stop 2024-03-16T10:11:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.156+0000 7f2ba2993640 1 -- 172.21.15.17:0/1205194632 shutdown_connections 2024-03-16T10:11:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.156+0000 7f2ba2993640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2b88067660 0x7f2b88069b20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.156+0000 7f2ba2993640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2b9c072b10 0x7f2b9c1de210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.156+0000 7f2ba2993640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b9c06fcd0 0x7f2b9c1ddcd0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.156+0000 7f2ba2993640 1 --2- 172.21.15.17:0/1205194632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b9c06f870 0x7f2b9c13d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.156+0000 7f2ba2993640 1 -- 172.21.15.17:0/1205194632 >> 172.21.15.17:0/1205194632 conn(0x7f2b9c0b77f0 msgr2=0x7f2b9c06edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.156+0000 7f2ba2993640 1 -- 172.21.15.17:0/1205194632 shutdown_connections 2024-03-16T10:11:27.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.156+0000 7f2ba2993640 1 -- 172.21.15.17:0/1205194632 wait complete. 2024-03-16T10:11:27.182 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:11:27.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.296+0000 7f1a1ee05640 1 Processor -- start 2024-03-16T10:11:27.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.296+0000 7f1a1ee05640 1 -- start start 2024-03-16T10:11:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a1ee05640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1a180b60e0 0x7f1a1813e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a1ee05640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1a180aed90 0x7f1a1814ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a1ee05640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a1806dbf0 con 0x7f1a1806d810 2024-03-16T10:11:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a1ee05640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f1a180b0190 con 0x7f1a180aed90 2024-03-16T10:11:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a1ee05640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f1a180b5b10 con 0x7f1a180b60e0 2024-03-16T10:11:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a1d3a2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1a180aed90 0x7f1a1814ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a1d3a2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1a180aed90 0x7f1a1814ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55540/0 (socket says 172.21.15.17:55540) 2024-03-16T10:11:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a1d3a2640 1 -- 172.21.15.17:0/2854460772 learned_addr learned my addr 172.21.15.17:0/2854460772 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a0ffff640 1 --2- 172.21.15.17:0/2854460772 >> v2:172.21.15.174:3300/0 conn(0x7f1a180b60e0 0x7f1a1813e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a1d3a2640 1 -- 172.21.15.17:0/2854460772 >> v2:172.21.15.174:3300/0 conn(0x7f1a180b60e0 msgr2=0x7f1a1813e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:27.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a1d3a2640 1 --2- 172.21.15.17:0/2854460772 >> v2:172.21.15.174:3300/0 conn(0x7f1a180b60e0 0x7f1a1813e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a1d3a2640 1 -- 172.21.15.17:0/2854460772 >> v1:172.21.15.174:6789/0 conn(0x7f1a1806d810 legacy=0x7f1a180af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:27.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a1d3a2640 1 -- 172.21.15.17:0/2854460772 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a180afc70 con 0x7f1a180aed90 2024-03-16T10:11:27.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a1d3a2640 1 --2- 172.21.15.17:0/2854460772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a180aed90 0x7f1a1814ea70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1a180b75b0 tx=0x7f1a1001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c81c130c35e5a78 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:27.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a0f7fe640 1 -- 172.21.15.17:0/2854460772 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a100a56c0 con 0x7f1a180aed90 2024-03-16T10:11:27.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a0f7fe640 1 -- 172.21.15.17:0/2854460772 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a100a5860 con 0x7f1a180aed90 2024-03-16T10:11:27.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.300+0000 7f1a0f7fe640 1 -- 172.21.15.17:0/2854460772 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a100a5c30 con 0x7f1a180aed90 2024-03-16T10:11:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2854460772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a180aed90 msgr2=0x7f1a1814ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 --2- 172.21.15.17:0/2854460772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a180aed90 0x7f1a1814ea70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1a180b75b0 tx=0x7f1a1001c8a0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2854460772 shutdown_connections 2024-03-16T10:11:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 --2- 172.21.15.17:0/2854460772 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a180aed90 0x7f1a1814ea70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 --2- 172.21.15.17:0/2854460772 >> v2:172.21.15.174:3300/0 conn(0x7f1a180b60e0 0x7f1a1813e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2854460772 >> 172.21.15.17:0/2854460772 conn(0x7f1a180b9a30 msgr2=0x7f1a180b9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2854460772 shutdown_connections 2024-03-16T10:11:27.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2854460772 wait complete. 2024-03-16T10:11:27.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 Processor -- start 2024-03-16T10:11:27.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 -- start start 2024-03-16T10:11:27.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1a1806d810 0x7f1a181e3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a180aed90 0x7f1a1809f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a180b60e0 0x7f1a1813b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1a18151b60 con 0x7f1a180aed90 2024-03-16T10:11:27.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a181519e0 con 0x7f1a180b60e0 2024-03-16T10:11:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1a18151ce0 con 0x7f1a1806d810 2024-03-16T10:11:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a0ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a180aed90 0x7f1a1809f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a0ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a180aed90 0x7f1a1809f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55544/0 (socket says 172.21.15.17:55544) 2024-03-16T10:11:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a0ffff640 1 -- 172.21.15.17:0/2262003192 learned_addr learned my addr 172.21.15.17:0/2262003192 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1d3a2640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a180b60e0 0x7f1a1813b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a0ffff640 1 -- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1a1806d810 msgr2=0x7f1a181e3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1cba1640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1a1806d810 0x7f1a181e3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a0ffff640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1a1806d810 0x7f1a181e3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a0ffff640 1 -- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a180b60e0 msgr2=0x7f1a1813b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a0ffff640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a180b60e0 0x7f1a1813b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a0ffff640 1 -- 172.21.15.17:0/2262003192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a181346e0 con 0x7f1a180aed90 2024-03-16T10:11:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1d3a2640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a180b60e0 0x7f1a1813b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1cba1640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1a1806d810 0x7f1a181e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:27.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a0ffff640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a180aed90 0x7f1a1809f130 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800d800 tx=0x7f1a0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a0d7fa640 1 -- 172.21.15.17:0/2262003192 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800b6c0 con 0x7f1a180aed90 2024-03-16T10:11:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a0d7fa640 1 -- 172.21.15.17:0/2262003192 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a08019070 con 0x7f1a180aed90 2024-03-16T10:11:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2262003192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a18137fc0 con 0x7f1a180aed90 2024-03-16T10:11:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a0d7fa640 1 -- 172.21.15.17:0/2262003192 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800bae0 con 0x7f1a180aed90 2024-03-16T10:11:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.304+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2262003192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a181349c0 con 0x7f1a180aed90 2024-03-16T10:11:27.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.308+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2262003192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a181344d0 con 0x7f1a180aed90 2024-03-16T10:11:27.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.308+0000 7f1a0d7fa640 1 -- 172.21.15.17:0/2262003192 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f1a0800bc80 con 0x7f1a180aed90 2024-03-16T10:11:27.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.308+0000 7f1a0d7fa640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1a04067660 0x7f1a04069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.308+0000 7f1a0d7fa640 1 -- 172.21.15.17:0/2262003192 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1a08085d70 con 0x7f1a180aed90 2024-03-16T10:11:27.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.308+0000 7f1a1cba1640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1a04067660 0x7f1a04069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.308+0000 7f1a1cba1640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1a04067660 0x7f1a04069b20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a14001650 tx=0x7f1a14009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:27.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.312+0000 7f1a0d7fa640 1 -- 172.21.15.17:0/2262003192 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1a08056a80 con 0x7f1a180aed90 2024-03-16T10:11:27.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.532+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2262003192 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a1806a130 con 0x7f1a180aed90 2024-03-16T10:11:27.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.532+0000 7f1a0d7fa640 1 -- 172.21.15.17:0/2262003192 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7f1a08056380 con 0x7f1a180aed90 2024-03-16T10:11:27.537 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:27.538 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1058712158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1058712158},{"type":"v1","addr":"172.21.15.174:6835","nonce":1058712158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:10:57.464714+0000","modified":"2024-03-16T10:11:00.509595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6833/3105182870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3105182870},{"type":"v1","addr":"172.21.15.17:6833","nonce":3105182870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-16T10:11:27.538 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 8 2024-03-16T10:11:27.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.532+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1a04067660 msgr2=0x7f1a04069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.532+0000 7f1a1ee05640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1a04067660 0x7f1a04069b20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a14001650 tx=0x7f1a14009040 comp rx=0 tx=0).stop 2024-03-16T10:11:27.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.532+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2262003192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a180aed90 msgr2=0x7f1a1809f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.532+0000 7f1a1ee05640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a180aed90 0x7f1a1809f130 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800d800 tx=0x7f1a0800dcc0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.532+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2262003192 shutdown_connections 2024-03-16T10:11:27.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.536+0000 7f1a1ee05640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1a04067660 0x7f1a04069b20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.536+0000 7f1a1ee05640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a180b60e0 0x7f1a1813b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.536+0000 7f1a1ee05640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1a180aed90 0x7f1a1809f130 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.536+0000 7f1a1ee05640 1 --2- 172.21.15.17:0/2262003192 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1a1806d810 0x7f1a181e3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.536+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2262003192 >> 172.21.15.17:0/2262003192 conn(0x7f1a180b9a30 msgr2=0x7f1a180b4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:27.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.536+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2262003192 shutdown_connections 2024-03-16T10:11:27.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.536+0000 7f1a1ee05640 1 -- 172.21.15.17:0/2262003192 wait complete. 2024-03-16T10:11:27.561 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:27.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.680+0000 7f8f86211640 1 Processor -- start 2024-03-16T10:11:27.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.680+0000 7f8f86211640 1 -- start start 2024-03-16T10:11:27.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f86211640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f800b0f60 0x7f8f800b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f86211640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f80073230 con 0x7f8f8014dbd0 2024-03-16T10:11:27.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f86211640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f800b69b0 con 0x7f8f8006ede0 2024-03-16T10:11:27.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f86211640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8f80142e00 con 0x7f8f800b0f60 2024-03-16T10:11:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7ffff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f8f8014dbd0 0x7f8f8014ffc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46192/0 (socket says 172.21.15.17:46192) 2024-03-16T10:11:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7ffff640 1 -- 172.21.15.17:0/680675524 learned_addr learned my addr 172.21.15.17:0/680675524 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7effd640 1 --2- 172.21.15.17:0/680675524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f800b0f60 0x7f8f800b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 -- 172.21.15.17:0/680675524 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3650633259 0 0) 0x7f8f80073230 con 0x7f8f8014dbd0 2024-03-16T10:11:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 -- 172.21.15.17:0/680675524 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f6c003530 con 0x7f8f8014dbd0 2024-03-16T10:11:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 -- 172.21.15.17:0/680675524 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1652135676 0 0) 0x7f8f6c003530 con 0x7f8f8014dbd0 2024-03-16T10:11:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 -- 172.21.15.17:0/680675524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f800b0f60 msgr2=0x7f8f800b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 --2- 172.21.15.17:0/680675524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f800b0f60 0x7f8f800b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 -- 172.21.15.17:0/680675524 >> v1:172.21.15.174:6789/0 conn(0x7f8f8006ede0 legacy=0x7f8f80145340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 -- 172.21.15.17:0/680675524 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8006f1c0 con 0x7f8f8014dbd0 2024-03-16T10:11:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 -- 172.21.15.17:0/680675524 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f8f78001680 con 0x7f8f8014dbd0 2024-03-16T10:11:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 -- 172.21.15.17:0/680675524 >> v1:172.21.15.17:6789/0 conn(0x7f8f8014dbd0 legacy=0x7f8f8014ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 --2- 172.21.15.17:0/680675524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f800b0f60 0x7f8f6c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 --2- 172.21.15.17:0/680675524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f6c003f30 0x7f8f6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 --2- 172.21.15.17:0/680675524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c004d50 0x7f8f6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 -- 172.21.15.17:0/680675524 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8f80142e00 con 0x7f8f6c004d50 2024-03-16T10:11:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 -- 172.21.15.17:0/680675524 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6c00bae0 con 0x7f8f6c003f30 2024-03-16T10:11:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 -- 172.21.15.17:0/680675524 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6c00bce0 con 0x7f8f800b0f60 2024-03-16T10:11:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7ffff640 1 --2- 172.21.15.17:0/680675524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c004d50 0x7f8f6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7f7fe640 1 --2- 172.21.15.17:0/680675524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f800b0f60 0x7f8f6c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7effd640 1 --2- 172.21.15.17:0/680675524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f6c003f30 0x7f8f6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7ffff640 1 -- 172.21.15.17:0/680675524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f800b0f60 msgr2=0x7f8f6c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7ffff640 1 --2- 172.21.15.17:0/680675524 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f800b0f60 0x7f8f6c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7ffff640 1 -- 172.21.15.17:0/680675524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f6c003f30 msgr2=0x7f8f6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7ffff640 1 --2- 172.21.15.17:0/680675524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f6c003f30 0x7f8f6c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7ffff640 1 -- 172.21.15.17:0/680675524 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6c003530 con 0x7f8f6c004d50 2024-03-16T10:11:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7ffff640 1 --2- 172.21.15.17:0/680675524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c004d50 0x7f8f6c0051d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800c2c0 tx=0x7f8f7801cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440c92489a7eaedb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 -- 172.21.15.17:0/680675524 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78007d60 con 0x7f8f6c004d50 2024-03-16T10:11:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 -- 172.21.15.17:0/680675524 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f780a9070 con 0x7f8f6c004d50 2024-03-16T10:11:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.684+0000 7f8f7e7fc640 1 -- 172.21.15.17:0/680675524 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f78005d80 con 0x7f8f6c004d50 2024-03-16T10:11:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 -- 172.21.15.17:0/680675524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c004d50 msgr2=0x7f8f6c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 --2- 172.21.15.17:0/680675524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c004d50 0x7f8f6c0051d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8f7800c2c0 tx=0x7f8f7801cd20 comp rx=0 tx=0).stop 2024-03-16T10:11:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f7ffff640 1 -- 172.21.15.17:0/680675524 reap_dead start 2024-03-16T10:11:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 -- 172.21.15.17:0/680675524 shutdown_connections 2024-03-16T10:11:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 -- 172.21.15.17:0/680675524 >> 172.21.15.17:0/680675524 conn(0x7f8f800b78f0 msgr2=0x7f8f800b7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 -- 172.21.15.17:0/680675524 shutdown_connections 2024-03-16T10:11:27.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 -- 172.21.15.17:0/680675524 wait complete. 2024-03-16T10:11:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 Processor -- start 2024-03-16T10:11:27.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 -- start start 2024-03-16T10:11:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c004d50 0x7f8f8009c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f800b54f0 0x7f8f801e08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8009d980 0x7f8f8013fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8f80145a50 con 0x7f8f6c004d50 2024-03-16T10:11:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8f80070150 con 0x7f8f8009d980 2024-03-16T10:11:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8f800b1e30 con 0x7f8f800b54f0 2024-03-16T10:11:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f7f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c004d50 0x7f8f8009c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f7f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c004d50 0x7f8f8009c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55568/0 (socket says 172.21.15.17:55568) 2024-03-16T10:11:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f7f7fe640 1 -- 172.21.15.17:0/2386031358 learned_addr learned my addr 172.21.15.17:0/2386031358 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f7effd640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f800b54f0 0x7f8f801e08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f7ffff640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8009d980 0x7f8f8013fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f7f7fe640 1 -- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f800b54f0 msgr2=0x7f8f801e08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f7f7fe640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f800b54f0 0x7f8f801e08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f7f7fe640 1 -- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8009d980 msgr2=0x7f8f8013fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f7f7fe640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8009d980 0x7f8f8013fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f7f7fe640 1 -- 172.21.15.17:0/2386031358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8009ec60 con 0x7f8f6c004d50 2024-03-16T10:11:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f7ffff640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8009d980 0x7f8f8013fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f7f7fe640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c004d50 0x7f8f8009c700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000fb60 tx=0x7f8f7000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f77fff640 1 -- 172.21.15.17:0/2386031358 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7000dbd0 con 0x7f8f6c004d50 2024-03-16T10:11:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f77fff640 1 -- 172.21.15.17:0/2386031358 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f70018070 con 0x7f8f6c004d50 2024-03-16T10:11:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 -- 172.21.15.17:0/2386031358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f800b5a10 con 0x7f8f6c004d50 2024-03-16T10:11:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f77fff640 1 -- 172.21.15.17:0/2386031358 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70003980 con 0x7f8f6c004d50 2024-03-16T10:11:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 -- 172.21.15.17:0/2386031358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f800b6230 con 0x7f8f6c004d50 2024-03-16T10:11:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.688+0000 7f8f86211640 1 -- 172.21.15.17:0/2386031358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f80068360 con 0x7f8f6c004d50 2024-03-16T10:11:27.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.692+0000 7f8f77fff640 1 -- 172.21.15.17:0/2386031358 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8f70003b20 con 0x7f8f6c004d50 2024-03-16T10:11:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.692+0000 7f8f77fff640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8f60067690 0x7f8f60069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.692+0000 7f8f77fff640 1 -- 172.21.15.17:0/2386031358 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8f70086680 con 0x7f8f6c004d50 2024-03-16T10:11:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.692+0000 7f8f7effd640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8f60067690 0x7f8f60069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.692+0000 7f8f7effd640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8f60067690 0x7f8f60069b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f800b5440 tx=0x7f8f68008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.696+0000 7f8f77fff640 1 -- 172.21.15.17:0/2386031358 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8f70057370 con 0x7f8f6c004d50 2024-03-16T10:11:27.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.852+0000 7f8f86211640 1 -- 172.21.15.17:0/2386031358 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f8006fb90 con 0x7f8f6c004d50 2024-03-16T10:11:27.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.852+0000 7f8f77fff640 1 -- 172.21.15.17:0/2386031358 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7f8f70056c70 con 0x7f8f6c004d50 2024-03-16T10:11:27.859 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:27.859 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.856+0000 7f8f86211640 1 -- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8f60067690 msgr2=0x7f8f60069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.856+0000 7f8f86211640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8f60067690 0x7f8f60069b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8f800b5440 tx=0x7f8f68008040 comp rx=0 tx=0).stop 2024-03-16T10:11:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.856+0000 7f8f86211640 1 -- 172.21.15.17:0/2386031358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c004d50 msgr2=0x7f8f8009c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.856+0000 7f8f86211640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c004d50 0x7f8f8009c700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000fb60 tx=0x7f8f7000d520 comp rx=0 tx=0).stop 2024-03-16T10:11:27.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.856+0000 7f8f86211640 1 -- 172.21.15.17:0/2386031358 shutdown_connections 2024-03-16T10:11:27.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.856+0000 7f8f86211640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8f60067690 0x7f8f60069b50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.856+0000 7f8f86211640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8009d980 0x7f8f8013fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.856+0000 7f8f86211640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8f800b54f0 0x7f8f801e08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.856+0000 7f8f86211640 1 --2- 172.21.15.17:0/2386031358 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f6c004d50 0x7f8f8009c700 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.856+0000 7f8f86211640 1 -- 172.21.15.17:0/2386031358 >> 172.21.15.17:0/2386031358 conn(0x7f8f800b78f0 msgr2=0x7f8f801432b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:27.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.856+0000 7f8f86211640 1 -- 172.21.15.17:0/2386031358 shutdown_connections 2024-03-16T10:11:27.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.856+0000 7f8f86211640 1 -- 172.21.15.17:0/2386031358 wait complete. 2024-03-16T10:11:27.877 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-16T10:11:27.877 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:11:27.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.980+0000 7f280c4ed640 1 Processor -- start 2024-03-16T10:11:27.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.980+0000 7f280c4ed640 1 -- start start 2024-03-16T10:11:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.984+0000 7f280c4ed640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2804072b80 0x7f28040b7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.984+0000 7f280c4ed640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f280413f1a0 0x7f280413f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.984+0000 7f280c4ed640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2804070290 con 0x7f2804145900 2024-03-16T10:11:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.984+0000 7f280c4ed640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f2804070570 con 0x7f280413f1a0 2024-03-16T10:11:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.984+0000 7f280c4ed640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f2804145440 con 0x7f2804072b80 2024-03-16T10:11:27.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.984+0000 7f2809a88640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2804145900 0x7f2804145d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46196/0 (socket says 172.21.15.17:46196) 2024-03-16T10:11:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.984+0000 7f280aa8a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f280413f1a0 0x7f280413f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.984+0000 7f2809a88640 1 -- 172.21.15.17:0/3472094697 learned_addr learned my addr 172.21.15.17:0/3472094697 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.984+0000 7f280a289640 1 --2- 172.21.15.17:0/3472094697 >> v2:172.21.15.174:3300/0 conn(0x7f2804072b80 0x7f28040b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.984+0000 7f280aa8a640 1 -- 172.21.15.17:0/3472094697 >> v2:172.21.15.174:3300/0 conn(0x7f2804072b80 msgr2=0x7f28040b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.984+0000 7f280aa8a640 1 --2- 172.21.15.17:0/3472094697 >> v2:172.21.15.174:3300/0 conn(0x7f2804072b80 0x7f28040b7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.984+0000 7f280aa8a640 1 -- 172.21.15.17:0/3472094697 >> v1:172.21.15.17:6789/0 conn(0x7f2804145900 legacy=0x7f2804145d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.984+0000 7f280aa8a640 1 -- 172.21.15.17:0/3472094697 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28040b7670 con 0x7f280413f1a0 2024-03-16T10:11:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.984+0000 7f280aa8a640 1 --2- 172.21.15.17:0/3472094697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f280413f1a0 0x7f280413f5f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f280406c270 tx=0x7f27fc01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72381959b96c55a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f2809287640 1 -- 172.21.15.17:0/3472094697 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0b2070 con 0x7f280413f1a0 2024-03-16T10:11:27.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f2809287640 1 -- 172.21.15.17:0/3472094697 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27fc0a57c0 con 0x7f280413f1a0 2024-03-16T10:11:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f2809287640 1 -- 172.21.15.17:0/3472094697 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0a5b90 con 0x7f280413f1a0 2024-03-16T10:11:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 -- 172.21.15.17:0/3472094697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f280413f1a0 msgr2=0x7f280413f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 --2- 172.21.15.17:0/3472094697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f280413f1a0 0x7f280413f5f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f280406c270 tx=0x7f27fc01c810 comp rx=0 tx=0).stop 2024-03-16T10:11:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 -- 172.21.15.17:0/3472094697 shutdown_connections 2024-03-16T10:11:27.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 --2- 172.21.15.17:0/3472094697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f280413f1a0 0x7f280413f5f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 --2- 172.21.15.17:0/3472094697 >> v2:172.21.15.174:3300/0 conn(0x7f2804072b80 0x7f28040b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 -- 172.21.15.17:0/3472094697 >> 172.21.15.17:0/3472094697 conn(0x7f28040b7880 msgr2=0x7f28040b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 -- 172.21.15.17:0/3472094697 shutdown_connections 2024-03-16T10:11:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 -- 172.21.15.17:0/3472094697 wait complete. 2024-03-16T10:11:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 Processor -- start 2024-03-16T10:11:27.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 -- start start 2024-03-16T10:11:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2804072b80 0x7f28040b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f280413f1a0 0x7f28041e0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2804145900 0x7f28041e1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2804152500 con 0x7f2804072b80 2024-03-16T10:11:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f280413fb30 con 0x7f280413f1a0 2024-03-16T10:11:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2804152680 con 0x7f2804145900 2024-03-16T10:11:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280a289640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2804072b80 0x7f28040b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280a289640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2804072b80 0x7f28040b3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55590/0 (socket says 172.21.15.17:55590) 2024-03-16T10:11:27.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280a289640 1 -- 172.21.15.17:0/1995257154 learned_addr learned my addr 172.21.15.17:0/1995257154 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280aa8a640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2804145900 0x7f28041e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f2809a88640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f280413f1a0 0x7f28041e0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280a289640 1 -- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2804145900 msgr2=0x7f28041e1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280a289640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2804145900 0x7f28041e1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280a289640 1 -- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f280413f1a0 msgr2=0x7f28041e0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280a289640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f280413f1a0 0x7f28041e0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280a289640 1 -- 172.21.15.17:0/1995257154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28040b49a0 con 0x7f2804072b80 2024-03-16T10:11:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280aa8a640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2804145900 0x7f28041e1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280a289640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2804072b80 0x7f28040b3260 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f27f400d800 tx=0x7f27f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f27faffd640 1 -- 172.21.15.17:0/1995257154 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f400b6c0 con 0x7f2804072b80 2024-03-16T10:11:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f27faffd640 1 -- 172.21.15.17:0/1995257154 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27f4019070 con 0x7f2804072b80 2024-03-16T10:11:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 -- 172.21.15.17:0/1995257154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28040b4ef0 con 0x7f2804072b80 2024-03-16T10:11:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f27faffd640 1 -- 172.21.15.17:0/1995257154 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f400ba60 con 0x7f2804072b80 2024-03-16T10:11:27.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.988+0000 7f280c4ed640 1 -- 172.21.15.17:0/1995257154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28040b4c10 con 0x7f2804072b80 2024-03-16T10:11:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.992+0000 7f280c4ed640 1 -- 172.21.15.17:0/1995257154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280406a130 con 0x7f2804072b80 2024-03-16T10:11:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.992+0000 7f27faffd640 1 -- 172.21.15.17:0/1995257154 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f27f400bc00 con 0x7f2804072b80 2024-03-16T10:11:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.992+0000 7f27faffd640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f27e8067660 0x7f27e8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.992+0000 7f27faffd640 1 -- 172.21.15.17:0/1995257154 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f27f4086740 con 0x7f2804072b80 2024-03-16T10:11:27.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.992+0000 7f2809a88640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f27e8067660 0x7f27e8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:27.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.992+0000 7f2809a88640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f27e8067660 0x7f27e8069b20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f28041e0e50 tx=0x7f2800009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:28.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:27.996+0000 7f27faffd640 1 -- 172.21.15.17:0/1995257154 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f27f4057530 con 0x7f2804072b80 2024-03-16T10:11:28.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.180+0000 7f280c4ed640 1 -- 172.21.15.17:0/1995257154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2804145220 con 0x7f2804072b80 2024-03-16T10:11:28.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.184+0000 7f27faffd640 1 -- 172.21.15.17:0/1995257154 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7f27f4056e30 con 0x7f2804072b80 2024-03-16T10:11:28.187 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:28.188 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1058712158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1058712158},{"type":"v1","addr":"172.21.15.174:6835","nonce":1058712158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:10:57.464714+0000","modified":"2024-03-16T10:11:00.509595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6833/3105182870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3105182870},{"type":"v1","addr":"172.21.15.17:6833","nonce":3105182870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-16T10:11:28.188 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 8 2024-03-16T10:11:28.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.184+0000 7f280c4ed640 1 -- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f27e8067660 msgr2=0x7f27e8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:28.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.184+0000 7f280c4ed640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f27e8067660 0x7f27e8069b20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f28041e0e50 tx=0x7f2800009040 comp rx=0 tx=0).stop 2024-03-16T10:11:28.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.184+0000 7f280c4ed640 1 -- 172.21.15.17:0/1995257154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2804072b80 msgr2=0x7f28040b3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:28.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.184+0000 7f280c4ed640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2804072b80 0x7f28040b3260 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f27f400d800 tx=0x7f27f400dcc0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.184+0000 7f280c4ed640 1 -- 172.21.15.17:0/1995257154 shutdown_connections 2024-03-16T10:11:28.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.184+0000 7f280c4ed640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f27e8067660 0x7f27e8069b20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.184+0000 7f280c4ed640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2804145900 0x7f28041e1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.184+0000 7f280c4ed640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f280413f1a0 0x7f28041e0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.184+0000 7f280c4ed640 1 --2- 172.21.15.17:0/1995257154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2804072b80 0x7f28040b3260 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.184+0000 7f280c4ed640 1 -- 172.21.15.17:0/1995257154 >> 172.21.15.17:0/1995257154 conn(0x7f28040b7880 msgr2=0x7f28040b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:28.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.184+0000 7f280c4ed640 1 -- 172.21.15.17:0/1995257154 shutdown_connections 2024-03-16T10:11:28.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.184+0000 7f280c4ed640 1 -- 172.21.15.17:0/1995257154 wait complete. 2024-03-16T10:11:28.206 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:11:28.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.320+0000 7fe7df728640 1 Processor -- start 2024-03-16T10:11:28.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.320+0000 7fe7df728640 1 -- start start 2024-03-16T10:11:28.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7df728640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7d806e310 0x7fe7d806b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:28.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7df728640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fe7d80731f0 0x7fe7d8147440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:28.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7df728640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7d806e6f0 con 0x7fe7d80b0720 2024-03-16T10:11:28.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7df728640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe7d80b5d30 con 0x7fe7d806e310 2024-03-16T10:11:28.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7df728640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fe7d806b8b0 con 0x7fe7d80731f0 2024-03-16T10:11:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7dd4c4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7d806e310 0x7fe7d806b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7dd4c4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe7d806e310 0x7fe7d806b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55594/0 (socket says 172.21.15.17:55594) 2024-03-16T10:11:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7dd4c4640 1 -- 172.21.15.17:0/271730404 learned_addr learned my addr 172.21.15.17:0/271730404 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7dccc3640 1 --2- 172.21.15.17:0/271730404 >> v2:172.21.15.174:3300/0 conn(0x7fe7d80731f0 0x7fe7d8147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7dd4c4640 1 -- 172.21.15.17:0/271730404 >> v2:172.21.15.174:3300/0 conn(0x7fe7d80731f0 msgr2=0x7fe7d8147440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7dd4c4640 1 --2- 172.21.15.17:0/271730404 >> v2:172.21.15.174:3300/0 conn(0x7fe7d80731f0 0x7fe7d8147440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7dd4c4640 1 -- 172.21.15.17:0/271730404 >> v1:172.21.15.174:6789/0 conn(0x7fe7d80b0720 legacy=0x7fe7d814a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7dd4c4640 1 -- 172.21.15.17:0/271730404 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d806d5d0 con 0x7fe7d806e310 2024-03-16T10:11:28.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7dccc3640 1 --2- 172.21.15.17:0/271730404 >> v2:172.21.15.174:3300/0 conn(0x7fe7d80731f0 0x7fe7d8147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:28.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7dd4c4640 1 --2- 172.21.15.17:0/271730404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d806e310 0x7fe7d806b290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe7d806de70 tx=0x7fe7d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdc2be7a01f3b0e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:28.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7cffff640 1 -- 172.21.15.17:0/271730404 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40b0070 con 0x7fe7d806e310 2024-03-16T10:11:28.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7cffff640 1 -- 172.21.15.17:0/271730404 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d40ab600 con 0x7fe7d806e310 2024-03-16T10:11:28.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7cffff640 1 -- 172.21.15.17:0/271730404 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40ab9d0 con 0x7fe7d806e310 2024-03-16T10:11:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7df728640 1 -- 172.21.15.17:0/271730404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d806e310 msgr2=0x7fe7d806b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7df728640 1 --2- 172.21.15.17:0/271730404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d806e310 0x7fe7d806b290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe7d806de70 tx=0x7fe7d401c450 comp rx=0 tx=0).stop 2024-03-16T10:11:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7df728640 1 -- 172.21.15.17:0/271730404 shutdown_connections 2024-03-16T10:11:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7df728640 1 --2- 172.21.15.17:0/271730404 >> v2:172.21.15.174:3300/0 conn(0x7fe7d80731f0 0x7fe7d8147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7df728640 1 --2- 172.21.15.17:0/271730404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d806e310 0x7fe7d806b290 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7df728640 1 -- 172.21.15.17:0/271730404 >> 172.21.15.17:0/271730404 conn(0x7fe7d80ba0e0 msgr2=0x7fe7d80ba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.324+0000 7fe7df728640 1 -- 172.21.15.17:0/271730404 shutdown_connections 2024-03-16T10:11:28.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7df728640 1 -- 172.21.15.17:0/271730404 wait complete. 2024-03-16T10:11:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7df728640 1 Processor -- start 2024-03-16T10:11:28.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7df728640 1 -- start start 2024-03-16T10:11:28.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7df728640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7d806e310 0x7fe7d8070a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:28.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7df728640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7d80731f0 0x7fe7d8071540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:28.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7df728640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d80b0720 0x7fe7d8138d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:28.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7df728640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d8154f10 con 0x7fe7d80b0720 2024-03-16T10:11:28.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7df728640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d80b63d0 con 0x7fe7d806e310 2024-03-16T10:11:28.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7df728640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe7d8155090 con 0x7fe7d80731f0 2024-03-16T10:11:28.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7ddcc5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d80b0720 0x7fe7d8138d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:28.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7ddcc5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d80b0720 0x7fe7d8138d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55604/0 (socket says 172.21.15.17:55604) 2024-03-16T10:11:28.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7ddcc5640 1 -- 172.21.15.17:0/2139186786 learned_addr learned my addr 172.21.15.17:0/2139186786 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:28.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7dccc3640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7d80731f0 0x7fe7d8071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:28.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7dd4c4640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7d806e310 0x7fe7d8070a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:28.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7ddcc5640 1 -- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7d80731f0 msgr2=0x7fe7d8071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:28.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7ddcc5640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7d80731f0 0x7fe7d8071540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7ddcc5640 1 -- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7d806e310 msgr2=0x7fe7d8070a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:28.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7ddcc5640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7d806e310 0x7fe7d8070a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7ddcc5640 1 -- 172.21.15.17:0/2139186786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d8071a80 con 0x7fe7d80b0720 2024-03-16T10:11:28.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7dccc3640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7d80731f0 0x7fe7d8071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:28.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7dd4c4640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7d806e310 0x7fe7d8070a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:28.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7ddcc5640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d80b0720 0x7fe7d8138d60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000c780 tx=0x7fe7d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:28.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7cdffb640 1 -- 172.21.15.17:0/2139186786 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d0004b70 con 0x7fe7d80b0720 2024-03-16T10:11:28.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7cdffb640 1 -- 172.21.15.17:0/2139186786 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7d0004d10 con 0x7fe7d80b0720 2024-03-16T10:11:28.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7df728640 1 -- 172.21.15.17:0/2139186786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d8146fa0 con 0x7fe7d80b0720 2024-03-16T10:11:28.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7cdffb640 1 -- 172.21.15.17:0/2139186786 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d000aa90 con 0x7fe7d80b0720 2024-03-16T10:11:28.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.328+0000 7fe7df728640 1 -- 172.21.15.17:0/2139186786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d8145170 con 0x7fe7d80b0720 2024-03-16T10:11:28.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.332+0000 7fe7df728640 1 -- 172.21.15.17:0/2139186786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d8147170 con 0x7fe7d80b0720 2024-03-16T10:11:28.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.332+0000 7fe7cdffb640 1 -- 172.21.15.17:0/2139186786 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe7d000ac30 con 0x7fe7d80b0720 2024-03-16T10:11:28.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.332+0000 7fe7cdffb640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe7bc067660 0x7fe7bc069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:28.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.332+0000 7fe7cdffb640 1 -- 172.21.15.17:0/2139186786 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe7d0085a40 con 0x7fe7d80b0720 2024-03-16T10:11:28.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.332+0000 7fe7dd4c4640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe7bc067660 0x7fe7bc069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:28.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.332+0000 7fe7dd4c4640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe7bc067660 0x7fe7bc069b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40ad000 tx=0x7fe7d4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:28.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.336+0000 7fe7cdffb640 1 -- 172.21.15.17:0/2139186786 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe7d0020070 con 0x7fe7d80b0720 2024-03-16T10:11:28.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.524+0000 7fe7df728640 1 -- 172.21.15.17:0/2139186786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7d806a130 con 0x7fe7d80b0720 2024-03-16T10:11:28.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.524+0000 7fe7cdffb640 1 -- 172.21.15.17:0/2139186786 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3530 (secure 0 0 0) 0x7fe7d0085e20 con 0x7fe7d80b0720 2024-03-16T10:11:28.530 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:28.531 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1058712158","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1058712158},{"type":"v1","addr":"172.21.15.174:6835","nonce":1058712158}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:10:57.464714+0000","modified":"2024-03-16T10:11:00.509595+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4263":{"gid":4263,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6833/3105182870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3105182870},{"type":"v1","addr":"172.21.15.17:6833","nonce":3105182870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4263,"qdb_cluster":[4263]},"id":1}]} 2024-03-16T10:11:28.531 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 8 2024-03-16T10:11:28.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.528+0000 7fe7df728640 1 -- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe7bc067660 msgr2=0x7fe7bc069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:28.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.528+0000 7fe7df728640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe7bc067660 0x7fe7bc069b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40ad000 tx=0x7fe7d4008040 comp rx=0 tx=0).stop 2024-03-16T10:11:28.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.528+0000 7fe7df728640 1 -- 172.21.15.17:0/2139186786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d80b0720 msgr2=0x7fe7d8138d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:28.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.528+0000 7fe7df728640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d80b0720 0x7fe7d8138d60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe7d000c780 tx=0x7fe7d000cc40 comp rx=0 tx=0).stop 2024-03-16T10:11:28.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.528+0000 7fe7df728640 1 -- 172.21.15.17:0/2139186786 shutdown_connections 2024-03-16T10:11:28.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.528+0000 7fe7df728640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe7bc067660 0x7fe7bc069b20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.528+0000 7fe7df728640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7d80b0720 0x7fe7d8138d60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.528+0000 7fe7df728640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe7d80731f0 0x7fe7d8071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.528+0000 7fe7df728640 1 --2- 172.21.15.17:0/2139186786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7d806e310 0x7fe7d8070a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.528+0000 7fe7df728640 1 -- 172.21.15.17:0/2139186786 >> 172.21.15.17:0/2139186786 conn(0x7fe7d80ba0e0 msgr2=0x7fe7d80b89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.528+0000 7fe7df728640 1 -- 172.21.15.17:0/2139186786 shutdown_connections 2024-03-16T10:11:28.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.528+0000 7fe7df728640 1 -- 172.21.15.17:0/2139186786 wait complete. 2024-03-16T10:11:28.548 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:28.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.668+0000 7fc95f6d1640 1 Processor -- start 2024-03-16T10:11:28.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.668+0000 7fc95f6d1640 1 -- start start 2024-03-16T10:11:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95f6d1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc9580b5970 0x7fc95806f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95f6d1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc95813f2c0 0x7fc95813f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95f6d1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9580b5d50 con 0x7fc958148a10 2024-03-16T10:11:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95f6d1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc95806da70 con 0x7fc9580b5970 2024-03-16T10:11:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95f6d1640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fc95806dd30 con 0x7fc95813f2c0 2024-03-16T10:11:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95d46d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc9580b5970 0x7fc95806f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95d46d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc9580b5970 0x7fc95806f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55610/0 (socket says 172.21.15.17:55610) 2024-03-16T10:11:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95d46d640 1 -- 172.21.15.17:0/2750631893 learned_addr learned my addr 172.21.15.17:0/2750631893 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95dc6e640 1 --1- 172.21.15.17:0/2750631893 >> v1:172.21.15.17:6789/0 conn(0x7fc958148a10 0x7fc958148e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:46210/0 (socket says 172.21.15.17:46210) 2024-03-16T10:11:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95cc6c640 1 --2- 172.21.15.17:0/2750631893 >> v2:172.21.15.174:3300/0 conn(0x7fc95813f2c0 0x7fc95813f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95d46d640 1 -- 172.21.15.17:0/2750631893 >> v2:172.21.15.174:3300/0 conn(0x7fc95813f2c0 msgr2=0x7fc95813f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95d46d640 1 --2- 172.21.15.17:0/2750631893 >> v2:172.21.15.174:3300/0 conn(0x7fc95813f2c0 0x7fc95813f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95d46d640 1 -- 172.21.15.17:0/2750631893 >> v1:172.21.15.17:6789/0 conn(0x7fc958148a10 legacy=0x7fc958148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:28.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95d46d640 1 -- 172.21.15.17:0/2750631893 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc958071af0 con 0x7fc9580b5970 2024-03-16T10:11:28.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc94bfff640 1 -- 172.21.15.17:0/2750631893 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3201691201 0 0) 0x7fc9580b5d50 con 0x7fc958148a10 2024-03-16T10:11:28.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc95d46d640 1 --2- 172.21.15.17:0/2750631893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9580b5970 0x7fc95806f0b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc95806ea40 tx=0x7fc94c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce29dc7f03f21dac server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:28.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc94bfff640 1 -- 172.21.15.17:0/2750631893 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0b0070 con 0x7fc9580b5970 2024-03-16T10:11:28.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc94bfff640 1 -- 172.21.15.17:0/2750631893 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc94c0ab600 con 0x7fc9580b5970 2024-03-16T10:11:28.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.672+0000 7fc94bfff640 1 -- 172.21.15.17:0/2750631893 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0ab9d0 con 0x7fc9580b5970 2024-03-16T10:11:28.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2750631893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9580b5970 msgr2=0x7fc95806f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:28.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 --2- 172.21.15.17:0/2750631893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9580b5970 0x7fc95806f0b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc95806ea40 tx=0x7fc94c01c400 comp rx=0 tx=0).stop 2024-03-16T10:11:28.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2750631893 shutdown_connections 2024-03-16T10:11:28.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 --2- 172.21.15.17:0/2750631893 >> v2:172.21.15.174:3300/0 conn(0x7fc95813f2c0 0x7fc95813f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 --2- 172.21.15.17:0/2750631893 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9580b5970 0x7fc95806f0b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2750631893 >> 172.21.15.17:0/2750631893 conn(0x7fc9580b6800 msgr2=0x7fc9580b6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:28.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2750631893 shutdown_connections 2024-03-16T10:11:28.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2750631893 wait complete. 2024-03-16T10:11:28.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 Processor -- start 2024-03-16T10:11:28.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 -- start start 2024-03-16T10:11:28.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9580b5970 0x7fc9581482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:28.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95813f2c0 0x7fc95813e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:28.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc958148a10 0x7fc9580b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:28.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc9581525c0 con 0x7fc958148a10 2024-03-16T10:11:28.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc958070a20 con 0x7fc95813f2c0 2024-03-16T10:11:28.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95f6d1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc958152740 con 0x7fc9580b5970 2024-03-16T10:11:28.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95dc6e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc958148a10 0x7fc9580b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:28.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95dc6e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc958148a10 0x7fc9580b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55624/0 (socket says 172.21.15.17:55624) 2024-03-16T10:11:28.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95dc6e640 1 -- 172.21.15.17:0/2564408883 learned_addr learned my addr 172.21.15.17:0/2564408883 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95d46d640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9580b5970 0x7fc9581482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95cc6c640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95813f2c0 0x7fc95813e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95dc6e640 1 -- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9580b5970 msgr2=0x7fc9581482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95dc6e640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9580b5970 0x7fc9581482a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95dc6e640 1 -- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95813f2c0 msgr2=0x7fc95813e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95dc6e640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95813f2c0 0x7fc95813e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95dc6e640 1 -- 172.21.15.17:0/2564408883 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9580b1e90 con 0x7fc958148a10 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95cc6c640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95813f2c0 0x7fc95813e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc95dc6e640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc958148a10 0x7fc9580b1950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc95000c700 tx=0x7fc95000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.676+0000 7fc949ffb640 1 -- 172.21.15.17:0/2564408883 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950018070 con 0x7fc958148a10 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.680+0000 7fc949ffb640 1 -- 172.21.15.17:0/2564408883 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95000ce10 con 0x7fc958148a10 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.680+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2564408883 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9580b1360 con 0x7fc958148a10 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.680+0000 7fc949ffb640 1 -- 172.21.15.17:0/2564408883 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95000a800 con 0x7fc958148a10 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.680+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2564408883 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc95813e9e0 con 0x7fc958148a10 2024-03-16T10:11:28.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.680+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2564408883 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9580b1750 con 0x7fc958148a10 2024-03-16T10:11:28.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.680+0000 7fc949ffb640 1 -- 172.21.15.17:0/2564408883 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fc95001b070 con 0x7fc958148a10 2024-03-16T10:11:28.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.680+0000 7fc949ffb640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc93c067660 0x7fc93c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:28.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.680+0000 7fc949ffb640 1 -- 172.21.15.17:0/2564408883 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc950084d10 con 0x7fc958148a10 2024-03-16T10:11:28.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.680+0000 7fc95d46d640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc93c067660 0x7fc93c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:28.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.680+0000 7fc95d46d640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc93c067660 0x7fc93c069b20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0ad000 tx=0x7fc94c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:28.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.684+0000 7fc949ffb640 1 -- 172.21.15.17:0/2564408883 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc9580b1750 con 0x7fc958148a10 2024-03-16T10:11:28.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.840+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2564408883 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc95806a130 con 0x7fc958148a10 2024-03-16T10:11:28.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.840+0000 7fc949ffb640 1 -- 172.21.15.17:0/2564408883 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16605 (secure 0 0 0) 0x7fc950055a20 con 0x7fc958148a10 2024-03-16T10:11:28.847 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:28.847 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":21,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:10:59.101774+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:10:55.723734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T10:10:56.459894+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:28.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.844+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc93c067660 msgr2=0x7fc93c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:28.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.844+0000 7fc95f6d1640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc93c067660 0x7fc93c069b20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0ad000 tx=0x7fc94c008040 comp rx=0 tx=0).stop 2024-03-16T10:11:28.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.844+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2564408883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc958148a10 msgr2=0x7fc9580b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:28.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.844+0000 7fc95f6d1640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc958148a10 0x7fc9580b1950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc95000c700 tx=0x7fc95000cbc0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.844+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2564408883 shutdown_connections 2024-03-16T10:11:28.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.844+0000 7fc95f6d1640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc93c067660 0x7fc93c069b20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.844+0000 7fc95f6d1640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc958148a10 0x7fc9580b1950 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.844+0000 7fc95f6d1640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc95813f2c0 0x7fc95813e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.844+0000 7fc95f6d1640 1 --2- 172.21.15.17:0/2564408883 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc9580b5970 0x7fc9581482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:28.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.844+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2564408883 >> 172.21.15.17:0/2564408883 conn(0x7fc9580b6800 msgr2=0x7fc9580b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:28.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.844+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2564408883 shutdown_connections 2024-03-16T10:11:28.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.844+0000 7fc95f6d1640 1 -- 172.21.15.17:0/2564408883 wait complete. 2024-03-16T10:11:28.864 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-16T10:11:28.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe4809d8640 1 Processor -- start 2024-03-16T10:11:28.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe4809d8640 1 -- start start 2024-03-16T10:11:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe4809d8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe47c067190 0x7fe47c13f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe4809d8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe47c06e000 0x7fe47c148750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe4809d8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe47c0b4c20 con 0x7fe47c13f610 2024-03-16T10:11:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe4809d8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe47c09bd00 con 0x7fe47c067190 2024-03-16T10:11:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe4809d8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe47c09bf50 con 0x7fe47c06e000 2024-03-16T10:11:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe47a59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe47c067190 0x7fe47c13f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe47a59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe47c067190 0x7fe47c13f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55636/0 (socket says 172.21.15.17:55636) 2024-03-16T10:11:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe47a59c640 1 -- 172.21.15.17:0/891119967 learned_addr learned my addr 172.21.15.17:0/891119967 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe479d9b640 1 --2- 172.21.15.17:0/891119967 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe47c06e000 0x7fe47c148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:28.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe47a59c640 1 -- 172.21.15.17:0/891119967 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe47c06e000 msgr2=0x7fe47c148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:29.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe47a59c640 1 --2- 172.21.15.17:0/891119967 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe47c06e000 0x7fe47c148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:29.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe47a59c640 1 -- 172.21.15.17:0/891119967 >> v1:172.21.15.17:6789/0 conn(0x7fe47c13f610 legacy=0x7fe47c1451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:29.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe47a59c640 1 -- 172.21.15.17:0/891119967 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47c0b4420 con 0x7fe47c067190 2024-03-16T10:11:29.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe479d9b640 1 --2- 172.21.15.17:0/891119967 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe47c06e000 0x7fe47c148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:29.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:28.992+0000 7fe47959a640 1 -- 172.21.15.17:0/891119967 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2649503958 0 0) 0x7fe47c0b4c20 con 0x7fe47c13f610 2024-03-16T10:11:29.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.000+0000 7fe47a59c640 1 --2- 172.21.15.17:0/891119967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47c067190 0x7fe47c13f0d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe474009e50 tx=0x7fe47401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3177ad588a4d1b1f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:29.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.000+0000 7fe47959a640 1 -- 172.21.15.17:0/891119967 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4740a56f0 con 0x7fe47c067190 2024-03-16T10:11:29.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.000+0000 7fe47959a640 1 -- 172.21.15.17:0/891119967 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4740a5890 con 0x7fe47c067190 2024-03-16T10:11:29.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.000+0000 7fe47959a640 1 -- 172.21.15.17:0/891119967 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4740a5c60 con 0x7fe47c067190 2024-03-16T10:11:29.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.000+0000 7fe4809d8640 1 -- 172.21.15.17:0/891119967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47c067190 msgr2=0x7fe47c13f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:29.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.000+0000 7fe4809d8640 1 --2- 172.21.15.17:0/891119967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47c067190 0x7fe47c13f0d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe474009e50 tx=0x7fe47401c9a0 comp rx=0 tx=0).stop 2024-03-16T10:11:29.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.000+0000 7fe4809d8640 1 -- 172.21.15.17:0/891119967 shutdown_connections 2024-03-16T10:11:29.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.000+0000 7fe4809d8640 1 --2- 172.21.15.17:0/891119967 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe47c06e000 0x7fe47c148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:29.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.000+0000 7fe4809d8640 1 --2- 172.21.15.17:0/891119967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47c067190 0x7fe47c13f0d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.000+0000 7fe4809d8640 1 -- 172.21.15.17:0/891119967 >> 172.21.15.17:0/891119967 conn(0x7fe47c0b5e10 msgr2=0x7fe47c0b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.000+0000 7fe4809d8640 1 -- 172.21.15.17:0/891119967 shutdown_connections 2024-03-16T10:11:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.000+0000 7fe4809d8640 1 -- 172.21.15.17:0/891119967 wait complete. 2024-03-16T10:11:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.000+0000 7fe4809d8640 1 Processor -- start 2024-03-16T10:11:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.000+0000 7fe4809d8640 1 -- start start 2024-03-16T10:11:29.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe4809d8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe47c06e000 0x7fe47c138cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:29.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe4809d8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe47c13f610 0x7fe47c1e1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:29.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe4809d8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47c13dbb0 0x7fe47c1e1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:29.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe4809d8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c152b00 con 0x7fe47c13dbb0 2024-03-16T10:11:29.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe4809d8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c0b6c20 con 0x7fe47c13f610 2024-03-16T10:11:29.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe4809d8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe47c152c80 con 0x7fe47c06e000 2024-03-16T10:11:29.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe47ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47c13dbb0 0x7fe47c1e1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:29.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe47ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47c13dbb0 0x7fe47c1e1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55642/0 (socket says 172.21.15.17:55642) 2024-03-16T10:11:29.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe47ad9d640 1 -- 172.21.15.17:0/4249901235 learned_addr learned my addr 172.21.15.17:0/4249901235 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:29.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe47a59c640 1 --2- 172.21.15.17:0/4249901235 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe47c06e000 0x7fe47c138cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:29.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe479d9b640 1 --2- 172.21.15.17:0/4249901235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe47c13f610 0x7fe47c1e1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:29.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe47ad9d640 1 -- 172.21.15.17:0/4249901235 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe47c06e000 msgr2=0x7fe47c138cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:29.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe47ad9d640 1 --2- 172.21.15.17:0/4249901235 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe47c06e000 0x7fe47c138cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:29.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe47ad9d640 1 -- 172.21.15.17:0/4249901235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe47c13f610 msgr2=0x7fe47c1e1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:29.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe47ad9d640 1 --2- 172.21.15.17:0/4249901235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe47c13f610 0x7fe47c1e1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:29.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe47ad9d640 1 -- 172.21.15.17:0/4249901235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47c137720 con 0x7fe47c13dbb0 2024-03-16T10:11:29.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe47ad9d640 1 --2- 172.21.15.17:0/4249901235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47c13dbb0 0x7fe47c1e1790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00c410 tx=0x7fe46c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:29.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe472ffd640 1 -- 172.21.15.17:0/4249901235 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c017070 con 0x7fe47c13dbb0 2024-03-16T10:11:29.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe472ffd640 1 -- 172.21.15.17:0/4249901235 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe46c003600 con 0x7fe47c13dbb0 2024-03-16T10:11:29.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe4809d8640 1 -- 172.21.15.17:0/4249901235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe47c13e040 con 0x7fe47c13dbb0 2024-03-16T10:11:29.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe472ffd640 1 -- 172.21.15.17:0/4249901235 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c0039f0 con 0x7fe47c13dbb0 2024-03-16T10:11:29.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe4809d8640 1 -- 172.21.15.17:0/4249901235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe47c146320 con 0x7fe47c13dbb0 2024-03-16T10:11:29.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe4809d8640 1 -- 172.21.15.17:0/4249901235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe47c13e430 con 0x7fe47c13dbb0 2024-03-16T10:11:29.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.004+0000 7fe472ffd640 1 -- 172.21.15.17:0/4249901235 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe46c01a070 con 0x7fe47c13dbb0 2024-03-16T10:11:29.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.008+0000 7fe472ffd640 1 --2- 172.21.15.17:0/4249901235 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe45c067660 0x7fe45c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:29.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.008+0000 7fe472ffd640 1 -- 172.21.15.17:0/4249901235 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe46c084c10 con 0x7fe47c13dbb0 2024-03-16T10:11:29.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.008+0000 7fe47a59c640 1 --2- 172.21.15.17:0/4249901235 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe45c067660 0x7fe45c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:29.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.008+0000 7fe47a59c640 1 --2- 172.21.15.17:0/4249901235 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe45c067660 0x7fe45c069b20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe4740ae000 tx=0x7fe4740a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:29.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.012+0000 7fe472ffd640 1 -- 172.21.15.17:0/4249901235 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe46c0559e0 con 0x7fe47c13dbb0 2024-03-16T10:11:29.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.208+0000 7fe4809d8640 1 -- 172.21.15.17:0/4249901235 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe47c06a130 con 0x7fe47c13dbb0 2024-03-16T10:11:29.213 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-16T10:11:29.208+0000 7f381f0a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:11:29.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.208+0000 7fe472ffd640 1 -- 172.21.15.17:0/4249901235 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7fe46c0552e0 con 0x7fe47c13dbb0 2024-03-16T10:11:29.215 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-16T10:11:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.212+0000 7fe4809d8640 1 -- 172.21.15.17:0/4249901235 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe45c067660 msgr2=0x7fe45c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.212+0000 7fe4809d8640 1 --2- 172.21.15.17:0/4249901235 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe45c067660 0x7fe45c069b20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe4740ae000 tx=0x7fe4740a7070 comp rx=0 tx=0).stop 2024-03-16T10:11:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.212+0000 7fe4809d8640 1 -- 172.21.15.17:0/4249901235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47c13dbb0 msgr2=0x7fe47c1e1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.212+0000 7fe4809d8640 1 --2- 172.21.15.17:0/4249901235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47c13dbb0 0x7fe47c1e1790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00c410 tx=0x7fe46c00c8d0 comp rx=0 tx=0).stop 2024-03-16T10:11:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.212+0000 7fe4809d8640 1 -- 172.21.15.17:0/4249901235 shutdown_connections 2024-03-16T10:11:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.212+0000 7fe4809d8640 1 --2- 172.21.15.17:0/4249901235 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe45c067660 0x7fe45c069b20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.212+0000 7fe4809d8640 1 --2- 172.21.15.17:0/4249901235 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe47c13dbb0 0x7fe47c1e1790 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.212+0000 7fe4809d8640 1 --2- 172.21.15.17:0/4249901235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe47c13f610 0x7fe47c1e1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.212+0000 7fe4809d8640 1 --2- 172.21.15.17:0/4249901235 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe47c06e000 0x7fe47c138cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.212+0000 7fe4809d8640 1 -- 172.21.15.17:0/4249901235 >> 172.21.15.17:0/4249901235 conn(0x7fe47c0b5e10 msgr2=0x7fe47c09d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.212+0000 7fe4809d8640 1 -- 172.21.15.17:0/4249901235 shutdown_connections 2024-03-16T10:11:29.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.212+0000 7fe4809d8640 1 -- 172.21.15.17:0/4249901235 wait complete. 2024-03-16T10:11:29.225 INFO:tasks.ceph.mds.a.smithi017.stderr: -1213> 2024-03-16T10:11:01.209+0000 7faf48b4c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-16T10:11:29.239 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-16T10:11:29.280 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-16T10:11:29.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.352+0000 7f181bde2640 1 Processor -- start 2024-03-16T10:11:29.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.352+0000 7f181bde2640 1 -- start start 2024-03-16T10:11:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.352+0000 7f181bde2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1814067220 0x7f18141488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.352+0000 7f181bde2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1814067600 con 0x7f1814148df0 2024-03-16T10:11:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.352+0000 7f181bde2640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18140b55d0 con 0x7f181409bd00 2024-03-16T10:11:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.352+0000 7f181bde2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f181409e820 con 0x7f1814067220 2024-03-16T10:11:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.352+0000 7f1819b7e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1814067220 0x7f18141488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.352+0000 7f181a37f640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1814148df0 0x7f181414fbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36130/0 (socket says 172.21.15.17:36130) 2024-03-16T10:11:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.352+0000 7f181a37f640 1 -- 172.21.15.17:0/3934139243 learned_addr learned my addr 172.21.15.17:0/3934139243 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.352+0000 7f1819b7e640 1 -- 172.21.15.17:0/3934139243 >> v1:172.21.15.174:6789/0 conn(0x7f181409bd00 legacy=0x7f181409d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.352+0000 7f1818b7c640 1 -- 172.21.15.17:0/3934139243 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3820916631 0 0) 0x7f1814067600 con 0x7f1814148df0 2024-03-16T10:11:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.352+0000 7f1819b7e640 1 -- 172.21.15.17:0/3934139243 >> v1:172.21.15.17:6789/0 conn(0x7f1814148df0 legacy=0x7f181414fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:29.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.352+0000 7f1819b7e640 1 -- 172.21.15.17:0/3934139243 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181409e540 con 0x7f1814067220 2024-03-16T10:11:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f1819b7e640 1 --2- 172.21.15.17:0/3934139243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1814067220 0x7f18141488b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f181409e9a0 tx=0x7f181001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b51c8c0bf6135a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f1818b7c640 1 -- 172.21.15.17:0/3934139243 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18100b2070 con 0x7f1814067220 2024-03-16T10:11:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f1818b7c640 1 -- 172.21.15.17:0/3934139243 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18100a5850 con 0x7f1814067220 2024-03-16T10:11:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f1818b7c640 1 -- 172.21.15.17:0/3934139243 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18100a5c20 con 0x7f1814067220 2024-03-16T10:11:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 -- 172.21.15.17:0/3934139243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1814067220 msgr2=0x7f18141488b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 --2- 172.21.15.17:0/3934139243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1814067220 0x7f18141488b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f181409e9a0 tx=0x7f181001c880 comp rx=0 tx=0).stop 2024-03-16T10:11:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 -- 172.21.15.17:0/3934139243 shutdown_connections 2024-03-16T10:11:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 --2- 172.21.15.17:0/3934139243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1814067220 0x7f18141488b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 -- 172.21.15.17:0/3934139243 >> 172.21.15.17:0/3934139243 conn(0x7f181406bd70 msgr2=0x7f18140b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 -- 172.21.15.17:0/3934139243 shutdown_connections 2024-03-16T10:11:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 -- 172.21.15.17:0/3934139243 wait complete. 2024-03-16T10:11:29.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 Processor -- start 2024-03-16T10:11:29.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 -- start start 2024-03-16T10:11:29.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1814067220 0x7f18140b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:29.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1814072a10 0x7f18140b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:29.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18140b36c0 0x7f1814143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:29.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f181409af00 con 0x7f1814072a10 2024-03-16T10:11:29.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f181409ad80 con 0x7f18140b36c0 2024-03-16T10:11:29.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181bde2640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1814153b20 con 0x7f1814067220 2024-03-16T10:11:29.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181937d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1814072a10 0x7f18140b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:29.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181937d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1814072a10 0x7f18140b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51012/0 (socket says 172.21.15.17:51012) 2024-03-16T10:11:29.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181937d640 1 -- 172.21.15.17:0/2576712489 learned_addr learned my addr 172.21.15.17:0/2576712489 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:29.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f1819b7e640 1 --2- 172.21.15.17:0/2576712489 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1814067220 0x7f18140b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:29.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181a37f640 1 --2- 172.21.15.17:0/2576712489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18140b36c0 0x7f1814143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:29.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181937d640 1 -- 172.21.15.17:0/2576712489 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1814067220 msgr2=0x7f18140b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:29.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181937d640 1 --2- 172.21.15.17:0/2576712489 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1814067220 0x7f18140b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:29.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181937d640 1 -- 172.21.15.17:0/2576712489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18140b36c0 msgr2=0x7f1814143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:29.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181937d640 1 --2- 172.21.15.17:0/2576712489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18140b36c0 0x7f1814143b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:29.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.356+0000 7f181937d640 1 -- 172.21.15.17:0/2576712489 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18140b28f0 con 0x7f1814072a10 2024-03-16T10:11:29.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.360+0000 7f181937d640 1 --2- 172.21.15.17:0/2576712489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1814072a10 0x7f18140b3180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f180400c450 tx=0x7f180400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:29.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.360+0000 7f180a7fc640 1 -- 172.21.15.17:0/2576712489 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804012830 con 0x7f1814072a10 2024-03-16T10:11:29.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.360+0000 7f180a7fc640 1 -- 172.21.15.17:0/2576712489 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1804017070 con 0x7f1814072a10 2024-03-16T10:11:29.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.360+0000 7f181bde2640 1 -- 172.21.15.17:0/2576712489 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18140b2bd0 con 0x7f1814072a10 2024-03-16T10:11:29.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.360+0000 7f180a7fc640 1 -- 172.21.15.17:0/2576712489 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804012c30 con 0x7f1814072a10 2024-03-16T10:11:29.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.360+0000 7f181bde2640 1 -- 172.21.15.17:0/2576712489 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18140b1810 con 0x7f1814072a10 2024-03-16T10:11:29.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.360+0000 7f181bde2640 1 -- 172.21.15.17:0/2576712489 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18140b3ad0 con 0x7f1814072a10 2024-03-16T10:11:29.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.360+0000 7f180a7fc640 1 -- 172.21.15.17:0/2576712489 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f180401c460 con 0x7f1814072a10 2024-03-16T10:11:29.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.360+0000 7f180a7fc640 1 --2- 172.21.15.17:0/2576712489 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f17f8067690 0x7f17f8069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:29.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.360+0000 7f180a7fc640 1 -- 172.21.15.17:0/2576712489 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f18040855e0 con 0x7f1814072a10 2024-03-16T10:11:29.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.360+0000 7f1819b7e640 1 --2- 172.21.15.17:0/2576712489 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f17f8067690 0x7f17f8069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:29.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.360+0000 7f1819b7e640 1 --2- 172.21.15.17:0/2576712489 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f17f8067690 0x7f17f8069b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f18100af000 tx=0x7f18100a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:29.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.364+0000 7f180a7fc640 1 -- 172.21.15.17:0/2576712489 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1804056290 con 0x7f1814072a10 2024-03-16T10:11:29.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:29.556+0000 7f181bde2640 1 -- 172.21.15.17:0/2576712489 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f181406a130 con 0x7f1814072a10 2024-03-16T10:11:29.739 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T10:11:29.742 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T10:11:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.284+0000 7f180a7fc640 1 -- 172.21.15.17:0/2576712489 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f1804055b90 con 0x7f1814072a10 2024-03-16T10:11:30.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.288+0000 7f181bde2640 1 -- 172.21.15.17:0/2576712489 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f17f8067690 msgr2=0x7f17f8069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:30.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.288+0000 7f181bde2640 1 --2- 172.21.15.17:0/2576712489 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f17f8067690 0x7f17f8069b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f18100af000 tx=0x7f18100a7070 comp rx=0 tx=0).stop 2024-03-16T10:11:30.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.288+0000 7f181bde2640 1 -- 172.21.15.17:0/2576712489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1814072a10 msgr2=0x7f18140b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:30.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.288+0000 7f181bde2640 1 --2- 172.21.15.17:0/2576712489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1814072a10 0x7f18140b3180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f180400c450 tx=0x7f180400c910 comp rx=0 tx=0).stop 2024-03-16T10:11:30.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.288+0000 7f181bde2640 1 -- 172.21.15.17:0/2576712489 shutdown_connections 2024-03-16T10:11:30.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.288+0000 7f181bde2640 1 --2- 172.21.15.17:0/2576712489 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f17f8067690 0x7f17f8069b50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.288+0000 7f181bde2640 1 --2- 172.21.15.17:0/2576712489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18140b36c0 0x7f1814143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.288+0000 7f181bde2640 1 --2- 172.21.15.17:0/2576712489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1814072a10 0x7f18140b3180 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.288+0000 7f181bde2640 1 --2- 172.21.15.17:0/2576712489 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1814067220 0x7f18140b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.288+0000 7f181bde2640 1 -- 172.21.15.17:0/2576712489 >> 172.21.15.17:0/2576712489 conn(0x7f181406bd70 msgr2=0x7f18140b7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.288+0000 7f181bde2640 1 -- 172.21.15.17:0/2576712489 shutdown_connections 2024-03-16T10:11:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.288+0000 7f181bde2640 1 -- 172.21.15.17:0/2576712489 wait complete. 2024-03-16T10:11:30.317 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-16T10:11:30.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50be4c5640 1 Processor -- start 2024-03-16T10:11:30.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50be4c5640 1 -- start start 2024-03-16T10:11:30.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50be4c5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50b80b2400 0x7f50b81452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:30.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50be4c5640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f50b80ae060 0x7f50b8072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:30.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50be4c5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50b80730c0 0x7f50b806c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:30.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50be4c5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f50b80afcf0 con 0x7f50b80b2400 2024-03-16T10:11:30.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50be4c5640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f50b80b3fe0 con 0x7f50b80ae060 2024-03-16T10:11:30.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50be4c5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f50b80af0c0 con 0x7f50b80730c0 2024-03-16T10:11:30.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50b7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50b80b2400 0x7f50b81452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:30.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50b7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f50b80b2400 0x7f50b81452c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51026/0 (socket says 172.21.15.17:51026) 2024-03-16T10:11:30.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50b7fff640 1 -- 172.21.15.17:0/3092483185 learned_addr learned my addr 172.21.15.17:0/3092483185 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:30.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50bca62640 1 --2- 172.21.15.17:0/3092483185 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50b80730c0 0x7f50b806c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:30.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50b77fe640 1 --2- 172.21.15.17:0/3092483185 >> v2:172.21.15.174:3300/0 conn(0x7f50b80ae060 0x7f50b8072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:30.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50b7fff640 1 -- 172.21.15.17:0/3092483185 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50b80730c0 msgr2=0x7f50b806c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:30.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50b7fff640 1 --2- 172.21.15.17:0/3092483185 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50b80730c0 0x7f50b806c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:30.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50b7fff640 1 -- 172.21.15.17:0/3092483185 >> v2:172.21.15.174:3300/0 conn(0x7f50b80ae060 msgr2=0x7f50b8072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:30.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50b7fff640 1 --2- 172.21.15.17:0/3092483185 >> v2:172.21.15.174:3300/0 conn(0x7f50b80ae060 0x7f50b8072b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:30.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50b7fff640 1 -- 172.21.15.17:0/3092483185 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b80b27e0 con 0x7f50b80b2400 2024-03-16T10:11:30.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.440+0000 7f50b77fe640 1 --2- 172.21.15.17:0/3092483185 >> v2:172.21.15.174:3300/0 conn(0x7f50b80ae060 0x7f50b8072b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:30.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.444+0000 7f50b7fff640 1 --2- 172.21.15.17:0/3092483185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50b80b2400 0x7f50b81452c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f50b80b41b0 tx=0x7f50a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6630540dc4fce577 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:30.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.444+0000 7f50b6ffd640 1 -- 172.21.15.17:0/3092483185 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a80aa660 con 0x7f50b80b2400 2024-03-16T10:11:30.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.444+0000 7f50b6ffd640 1 -- 172.21.15.17:0/3092483185 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50a80af070 con 0x7f50b80b2400 2024-03-16T10:11:30.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.444+0000 7f50b6ffd640 1 -- 172.21.15.17:0/3092483185 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a80aaa20 con 0x7f50b80b2400 2024-03-16T10:11:30.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.444+0000 7f50be4c5640 1 -- 172.21.15.17:0/3092483185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50b80b2400 msgr2=0x7f50b81452c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:30.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.444+0000 7f50be4c5640 1 --2- 172.21.15.17:0/3092483185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50b80b2400 0x7f50b81452c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f50b80b41b0 tx=0x7f50a801c520 comp rx=0 tx=0).stop 2024-03-16T10:11:30.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.444+0000 7f50be4c5640 1 -- 172.21.15.17:0/3092483185 shutdown_connections 2024-03-16T10:11:30.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.444+0000 7f50be4c5640 1 --2- 172.21.15.17:0/3092483185 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50b80730c0 0x7f50b806c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:30.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.444+0000 7f50be4c5640 1 --2- 172.21.15.17:0/3092483185 >> v2:172.21.15.174:3300/0 conn(0x7f50b80ae060 0x7f50b8072b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:30.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.444+0000 7f50be4c5640 1 --2- 172.21.15.17:0/3092483185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50b80b2400 0x7f50b81452c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:30.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.444+0000 7f50be4c5640 1 -- 172.21.15.17:0/3092483185 >> 172.21.15.17:0/3092483185 conn(0x7f50b80b8e90 msgr2=0x7f50b80b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:30.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.444+0000 7f50be4c5640 1 -- 172.21.15.17:0/3092483185 shutdown_connections 2024-03-16T10:11:30.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.444+0000 7f50be4c5640 1 -- 172.21.15.17:0/3092483185 wait complete. 2024-03-16T10:11:30.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50be4c5640 1 Processor -- start 2024-03-16T10:11:30.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50be4c5640 1 -- start start 2024-03-16T10:11:30.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50be4c5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50b80730c0 0x7f50b8140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:30.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50be4c5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50b80ae060 0x7f50b813dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:30.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50be4c5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50b80b2400 0x7f50b813e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:30.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50be4c5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f50b8145980 con 0x7f50b80ae060 2024-03-16T10:11:30.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50be4c5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50b8145800 con 0x7f50b80b2400 2024-03-16T10:11:30.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50be4c5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f50b8151bd0 con 0x7f50b80730c0 2024-03-16T10:11:30.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50b77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50b80ae060 0x7f50b813dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:30.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50b77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50b80ae060 0x7f50b813dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51030/0 (socket says 172.21.15.17:51030) 2024-03-16T10:11:30.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50b77fe640 1 -- 172.21.15.17:0/2829059924 learned_addr learned my addr 172.21.15.17:0/2829059924 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:30.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50bca62640 1 --2- 172.21.15.17:0/2829059924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50b80b2400 0x7f50b813e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:30.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50b7fff640 1 --2- 172.21.15.17:0/2829059924 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50b80730c0 0x7f50b8140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:30.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50b77fe640 1 -- 172.21.15.17:0/2829059924 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50b80730c0 msgr2=0x7f50b8140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:30.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50b77fe640 1 --2- 172.21.15.17:0/2829059924 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50b80730c0 0x7f50b8140dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:30.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50b77fe640 1 -- 172.21.15.17:0/2829059924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50b80b2400 msgr2=0x7f50b813e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:30.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50b77fe640 1 --2- 172.21.15.17:0/2829059924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50b80b2400 0x7f50b813e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:30.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50b77fe640 1 -- 172.21.15.17:0/2829059924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b809d2e0 con 0x7f50b80ae060 2024-03-16T10:11:30.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50b77fe640 1 --2- 172.21.15.17:0/2829059924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50b80ae060 0x7f50b813dd00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00c7f0 tx=0x7f50ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:30.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50b4ff9640 1 -- 172.21.15.17:0/2829059924 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac007a30 con 0x7f50b80ae060 2024-03-16T10:11:30.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50b4ff9640 1 -- 172.21.15.17:0/2829059924 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ac007bd0 con 0x7f50b80ae060 2024-03-16T10:11:30.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50b4ff9640 1 -- 172.21.15.17:0/2829059924 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac0113f0 con 0x7f50b80ae060 2024-03-16T10:11:30.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50be4c5640 1 -- 172.21.15.17:0/2829059924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b809d5c0 con 0x7f50b80ae060 2024-03-16T10:11:30.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50be4c5640 1 -- 172.21.15.17:0/2829059924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b80b1410 con 0x7f50b80ae060 2024-03-16T10:11:30.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50be4c5640 1 -- 172.21.15.17:0/2829059924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b809d0f0 con 0x7f50b80ae060 2024-03-16T10:11:30.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.448+0000 7f50b4ff9640 1 -- 172.21.15.17:0/2829059924 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f50ac011590 con 0x7f50b80ae060 2024-03-16T10:11:30.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.452+0000 7f50b4ff9640 1 --2- 172.21.15.17:0/2829059924 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f50a4067660 0x7f50a4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:30.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.452+0000 7f50b4ff9640 1 -- 172.21.15.17:0/2829059924 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f50ac085aa0 con 0x7f50b80ae060 2024-03-16T10:11:30.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.452+0000 7f50b7fff640 1 --2- 172.21.15.17:0/2829059924 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f50a4067660 0x7f50a4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:30.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.452+0000 7f50b7fff640 1 --2- 172.21.15.17:0/2829059924 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f50a4067660 0x7f50a4069b20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f50a80ae000 tx=0x7f50a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:30.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.456+0000 7f50b4ff9640 1 -- 172.21.15.17:0/2829059924 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f50ac015020 con 0x7f50b80ae060 2024-03-16T10:11:30.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:30.616+0000 7f50be4c5640 1 -- 172.21.15.17:0/2829059924 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f50b8068360 con 0x7f50b80ae060 2024-03-16T10:11:31.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.300+0000 7f50b4ff9640 1 -- 172.21.15.17:0/2829059924 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) ==== 181+0+0 (secure 0 0 0) 0x7f50ac056750 con 0x7f50b80ae060 2024-03-16T10:11:31.307 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-16T10:11:31.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.304+0000 7f50be4c5640 1 -- 172.21.15.17:0/2829059924 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f50a4067660 msgr2=0x7f50a4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:31.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.304+0000 7f50be4c5640 1 --2- 172.21.15.17:0/2829059924 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f50a4067660 0x7f50a4069b20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f50a80ae000 tx=0x7f50a8008040 comp rx=0 tx=0).stop 2024-03-16T10:11:31.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.304+0000 7f50be4c5640 1 -- 172.21.15.17:0/2829059924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50b80ae060 msgr2=0x7f50b813dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:31.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.304+0000 7f50be4c5640 1 --2- 172.21.15.17:0/2829059924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50b80ae060 0x7f50b813dd00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00c7f0 tx=0x7f50ac00ccb0 comp rx=0 tx=0).stop 2024-03-16T10:11:31.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.304+0000 7f50be4c5640 1 -- 172.21.15.17:0/2829059924 shutdown_connections 2024-03-16T10:11:31.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.304+0000 7f50be4c5640 1 --2- 172.21.15.17:0/2829059924 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f50a4067660 0x7f50a4069b20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:31.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.304+0000 7f50be4c5640 1 --2- 172.21.15.17:0/2829059924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50b80b2400 0x7f50b813e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:31.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.304+0000 7f50be4c5640 1 --2- 172.21.15.17:0/2829059924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50b80ae060 0x7f50b813dd00 secure :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00c7f0 tx=0x7f50ac00ccb0 comp rx=0 tx=0).stop 2024-03-16T10:11:31.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.304+0000 7f50be4c5640 1 --2- 172.21.15.17:0/2829059924 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f50b80730c0 0x7f50b8140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:31.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.304+0000 7f50be4c5640 1 -- 172.21.15.17:0/2829059924 >> 172.21.15.17:0/2829059924 conn(0x7f50b80b8e90 msgr2=0x7f50b80b5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:31.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.304+0000 7f50be4c5640 1 -- 172.21.15.17:0/2829059924 shutdown_connections 2024-03-16T10:11:31.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.304+0000 7f50be4c5640 1 -- 172.21.15.17:0/2829059924 wait complete. 2024-03-16T10:11:31.335 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-16T10:11:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68cdb3b640 1 Processor -- start 2024-03-16T10:11:31.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68cdb3b640 1 -- start start 2024-03-16T10:11:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68cdb3b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f68c80aeaf0 0x7f68c8071ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68cdb3b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68c8072410 0x7f68c814e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68cdb3b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68c80aef60 con 0x7f68c80b5740 2024-03-16T10:11:31.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68cdb3b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f68c806e950 con 0x7f68c80aeaf0 2024-03-16T10:11:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68cdb3b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f68c80b5b20 con 0x7f68c8072410 2024-03-16T10:11:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68c6ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f68c80aeaf0 0x7f68c8071ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68c77fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f68c80b5740 0x7f68c806ede0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36136/0 (socket says 172.21.15.17:36136) 2024-03-16T10:11:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68c77fe640 1 -- 172.21.15.17:0/171976773 learned_addr learned my addr 172.21.15.17:0/171976773 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68c6ffd640 1 -- 172.21.15.17:0/171976773 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68c8072410 msgr2=0x7f68c814e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68c7fff640 1 --2- 172.21.15.17:0/171976773 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68c8072410 0x7f68c814e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68c6ffd640 1 --2- 172.21.15.17:0/171976773 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68c8072410 0x7f68c814e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68c6ffd640 1 -- 172.21.15.17:0/171976773 >> v1:172.21.15.17:6789/0 conn(0x7f68c80b5740 legacy=0x7f68c806ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68c6ffd640 1 -- 172.21.15.17:0/171976773 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c80af720 con 0x7f68c80aeaf0 2024-03-16T10:11:31.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.464+0000 7f68c7fff640 1 --2- 172.21.15.17:0/171976773 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68c8072410 0x7f68c814e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68c6ffd640 1 --2- 172.21.15.17:0/171976773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c80aeaf0 0x7f68c8071ed0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0097c0 tx=0x7f68bc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333e9dc1dad497f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68c67fc640 1 -- 172.21.15.17:0/171976773 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0b0070 con 0x7f68c80aeaf0 2024-03-16T10:11:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68c67fc640 1 -- 172.21.15.17:0/171976773 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68bc0ab600 con 0x7f68c80aeaf0 2024-03-16T10:11:31.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68c67fc640 1 -- 172.21.15.17:0/171976773 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0ab9d0 con 0x7f68c80aeaf0 2024-03-16T10:11:31.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 -- 172.21.15.17:0/171976773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c80aeaf0 msgr2=0x7f68c8071ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 --2- 172.21.15.17:0/171976773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c80aeaf0 0x7f68c8071ed0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f68bc0097c0 tx=0x7f68bc01c4a0 comp rx=0 tx=0).stop 2024-03-16T10:11:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 -- 172.21.15.17:0/171976773 shutdown_connections 2024-03-16T10:11:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 --2- 172.21.15.17:0/171976773 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68c8072410 0x7f68c814e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 --2- 172.21.15.17:0/171976773 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c80aeaf0 0x7f68c8071ed0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 -- 172.21.15.17:0/171976773 >> 172.21.15.17:0/171976773 conn(0x7f68c80b65d0 msgr2=0x7f68c80b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 -- 172.21.15.17:0/171976773 shutdown_connections 2024-03-16T10:11:31.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 -- 172.21.15.17:0/171976773 wait complete. 2024-03-16T10:11:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 Processor -- start 2024-03-16T10:11:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 -- start start 2024-03-16T10:11:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c8072410 0x7f68c813d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c80aeaf0 0x7f68c81dcb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68c80b5740 0x7f68c81dd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:31.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81477a0 con 0x7f68c80aeaf0 2024-03-16T10:11:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68c8147620 con 0x7f68c8072410 2024-03-16T10:11:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68cdb3b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f68c8151710 con 0x7f68c80b5740 2024-03-16T10:11:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68c6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c80aeaf0 0x7f68c81dcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68c6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c80aeaf0 0x7f68c81dcb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51052/0 (socket says 172.21.15.17:51052) 2024-03-16T10:11:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.468+0000 7f68c6ffd640 1 -- 172.21.15.17:0/1981262756 learned_addr learned my addr 172.21.15.17:0/1981262756 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68c7fff640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68c80b5740 0x7f68c81dd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68c77fe640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c8072410 0x7f68c813d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68c6ffd640 1 -- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68c80b5740 msgr2=0x7f68c81dd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68c6ffd640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68c80b5740 0x7f68c81dd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68c6ffd640 1 -- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c8072410 msgr2=0x7f68c813d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68c6ffd640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c8072410 0x7f68c813d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68c6ffd640 1 -- 172.21.15.17:0/1981262756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c813bcd0 con 0x7f68c80aeaf0 2024-03-16T10:11:31.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68c7fff640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68c80b5740 0x7f68c81dd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:31.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68c77fe640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c8072410 0x7f68c813d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:31.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68c6ffd640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c80aeaf0 0x7f68c81dcb60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f68bc01a0d0 tx=0x7f68bc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:31.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68affff640 1 -- 172.21.15.17:0/1981262756 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0b0040 con 0x7f68c80aeaf0 2024-03-16T10:11:31.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68affff640 1 -- 172.21.15.17:0/1981262756 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68bc0b6020 con 0x7f68c80aeaf0 2024-03-16T10:11:31.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68cdb3b640 1 -- 172.21.15.17:0/1981262756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c813bfb0 con 0x7f68c80aeaf0 2024-03-16T10:11:31.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68affff640 1 -- 172.21.15.17:0/1981262756 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0b2600 con 0x7f68c80aeaf0 2024-03-16T10:11:31.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68cdb3b640 1 -- 172.21.15.17:0/1981262756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c813b3a0 con 0x7f68c80aeaf0 2024-03-16T10:11:31.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68cdb3b640 1 -- 172.21.15.17:0/1981262756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c813c5c0 con 0x7f68c80aeaf0 2024-03-16T10:11:31.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68affff640 1 -- 172.21.15.17:0/1981262756 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f68bc0b27a0 con 0x7f68c80aeaf0 2024-03-16T10:11:31.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68affff640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f68b4067690 0x7f68b4069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:31.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68affff640 1 -- 172.21.15.17:0/1981262756 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5357+0+0 (secure 0 0 0) 0x7f68bc120c50 con 0x7f68c80aeaf0 2024-03-16T10:11:31.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68c77fe640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f68b4067690 0x7f68b4069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:31.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.472+0000 7f68c77fe640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f68b4067690 0x7f68b4069b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f68c00012f0 tx=0x7f68c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:31.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.476+0000 7f68affff640 1 -- 172.21.15.17:0/1981262756 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f68bc126020 con 0x7f68c80aeaf0 2024-03-16T10:11:31.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:31.640+0000 7f68cdb3b640 1 -- 172.21.15.17:0/1981262756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f68c806a130 con 0x7f68c80aeaf0 2024-03-16T10:11:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.312+0000 7f68affff640 1 -- 172.21.15.17:0/1981262756 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) ==== 169+0+0 (secure 0 0 0) 0x7f68bc0c0030 con 0x7f68c80aeaf0 2024-03-16T10:11:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-16T10:11:32.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.316+0000 7f68cdb3b640 1 -- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f68b4067690 msgr2=0x7f68b4069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:32.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.316+0000 7f68cdb3b640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f68b4067690 0x7f68b4069b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f68c00012f0 tx=0x7f68c0009040 comp rx=0 tx=0).stop 2024-03-16T10:11:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.316+0000 7f68cdb3b640 1 -- 172.21.15.17:0/1981262756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c80aeaf0 msgr2=0x7f68c81dcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.316+0000 7f68cdb3b640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c80aeaf0 0x7f68c81dcb60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f68bc01a0d0 tx=0x7f68bc019240 comp rx=0 tx=0).stop 2024-03-16T10:11:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.316+0000 7f68cdb3b640 1 -- 172.21.15.17:0/1981262756 shutdown_connections 2024-03-16T10:11:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.316+0000 7f68cdb3b640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f68b4067690 0x7f68b4069b50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.316+0000 7f68cdb3b640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f68c80b5740 0x7f68c81dd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.316+0000 7f68cdb3b640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f68c80aeaf0 0x7f68c81dcb60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.316+0000 7f68cdb3b640 1 --2- 172.21.15.17:0/1981262756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c8072410 0x7f68c813d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.316+0000 7f68cdb3b640 1 -- 172.21.15.17:0/1981262756 >> 172.21.15.17:0/1981262756 conn(0x7f68c80b65d0 msgr2=0x7f68c80b7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.316+0000 7f68cdb3b640 1 -- 172.21.15.17:0/1981262756 shutdown_connections 2024-03-16T10:11:32.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.316+0000 7f68cdb3b640 1 -- 172.21.15.17:0/1981262756 wait complete. 2024-03-16T10:11:32.345 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-16T10:11:32.345 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-16T10:11:32.345 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-16T10:11:32.345 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:11:32.345 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-16T10:11:32.345 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-16T10:11:32.345 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-16T10:11:32.346 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:11:32.360 INFO:tasks.ceph.mds.a:Stopped 2024-03-16T10:11:32.360 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-16T10:11:32.371 INFO:tasks.ceph.mds.b:Stopped 2024-03-16T10:11:32.372 DEBUG:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-16T10:11:32.405 INFO:tasks.ceph.mds.a:Started 2024-03-16T10:11:32.405 DEBUG:teuthology.parallel:result is None 2024-03-16T10:11:32.416 INFO:tasks.ceph.mds.b:Started 2024-03-16T10:11:32.417 DEBUG:teuthology.parallel:result is None 2024-03-16T10:11:32.417 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-16T10:11:32.477 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-16T10:11:32.490 INFO:tasks.ceph.mds.b.smithi174.stdout:starting mds.b at 2024-03-16T10:11:32.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f11878dd640 1 Processor -- start 2024-03-16T10:11:32.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f11878dd640 1 -- start start 2024-03-16T10:11:32.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f11878dd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1180067220 0x7f11800b5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:32.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f11878dd640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f11801457f0 0x7f118014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:32.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f11878dd640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11800b4a60 con 0x7f1180145380 2024-03-16T10:11:32.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f11878dd640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f1180148970 con 0x7f1180067220 2024-03-16T10:11:32.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f11878dd640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f1180146360 con 0x7f11801457f0 2024-03-16T10:11:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f1184e78640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1180145380 0x7f118013f140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36148/0 (socket says 172.21.15.17:36148) 2024-03-16T10:11:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f1184e78640 1 -- 172.21.15.17:0/1526193171 learned_addr learned my addr 172.21.15.17:0/1526193171 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f1185679640 1 --2- 172.21.15.17:0/1526193171 >> v2:172.21.15.17:3300/0 conn(0x7f1180067220 0x7f11800b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f1185e7a640 1 --2- 172.21.15.17:0/1526193171 >> v2:172.21.15.174:3300/0 conn(0x7f11801457f0 0x7f118014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f1173fff640 1 -- 172.21.15.17:0/1526193171 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3633130432 0 0) 0x7f11800b4a60 con 0x7f1180145380 2024-03-16T10:11:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f1185679640 1 -- 172.21.15.17:0/1526193171 >> v2:172.21.15.174:3300/0 conn(0x7f11801457f0 msgr2=0x7f118014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f1185679640 1 --2- 172.21.15.17:0/1526193171 >> v2:172.21.15.174:3300/0 conn(0x7f11801457f0 0x7f118014fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f1185679640 1 -- 172.21.15.17:0/1526193171 >> v1:172.21.15.17:6789/0 conn(0x7f1180145380 legacy=0x7f118013f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:32.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.528+0000 7f1185679640 1 -- 172.21.15.17:0/1526193171 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1180067600 con 0x7f1180067220 2024-03-16T10:11:32.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.532+0000 7f1185679640 1 --2- 172.21.15.17:0/1526193171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1180067220 0x7f11800b5700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f11800b3e40 tx=0x7f117401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2964c5f3886d80a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.532+0000 7f1173fff640 1 -- 172.21.15.17:0/1526193171 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11740b0070 con 0x7f1180067220 2024-03-16T10:11:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.532+0000 7f1173fff640 1 -- 172.21.15.17:0/1526193171 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11740ab600 con 0x7f1180067220 2024-03-16T10:11:32.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.532+0000 7f1173fff640 1 -- 172.21.15.17:0/1526193171 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11740ab9d0 con 0x7f1180067220 2024-03-16T10:11:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.532+0000 7f11878dd640 1 -- 172.21.15.17:0/1526193171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1180067220 msgr2=0x7f11800b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.532+0000 7f11878dd640 1 --2- 172.21.15.17:0/1526193171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1180067220 0x7f11800b5700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f11800b3e40 tx=0x7f117401c3f0 comp rx=0 tx=0).stop 2024-03-16T10:11:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.532+0000 7f11878dd640 1 -- 172.21.15.17:0/1526193171 shutdown_connections 2024-03-16T10:11:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.532+0000 7f11878dd640 1 --2- 172.21.15.17:0/1526193171 >> v2:172.21.15.174:3300/0 conn(0x7f11801457f0 0x7f118014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.532+0000 7f11878dd640 1 --2- 172.21.15.17:0/1526193171 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1180067220 0x7f11800b5700 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.532+0000 7f11878dd640 1 -- 172.21.15.17:0/1526193171 >> 172.21.15.17:0/1526193171 conn(0x7f11800b9380 msgr2=0x7f11800b9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.532+0000 7f11878dd640 1 -- 172.21.15.17:0/1526193171 shutdown_connections 2024-03-16T10:11:32.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.532+0000 7f11878dd640 1 -- 172.21.15.17:0/1526193171 wait complete. 2024-03-16T10:11:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.532+0000 7f11878dd640 1 Processor -- start 2024-03-16T10:11:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.532+0000 7f11878dd640 1 -- start start 2024-03-16T10:11:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f11878dd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1180067220 0x7f11801404f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f11878dd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1180145380 0x7f118009eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:32.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f11878dd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11801e0ff0 0x7f1180140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f11878dd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f11801496a0 con 0x7f1180067220 2024-03-16T10:11:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f11878dd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1180149210 con 0x7f1180145380 2024-03-16T10:11:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f11878dd640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1180149820 con 0x7f11801e0ff0 2024-03-16T10:11:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1185679640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1180067220 0x7f11801404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1185679640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1180067220 0x7f11801404f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51078/0 (socket says 172.21.15.17:51078) 2024-03-16T10:11:32.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1185679640 1 -- 172.21.15.17:0/1943186112 learned_addr learned my addr 172.21.15.17:0/1943186112 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1185e7a640 1 --2- 172.21.15.17:0/1943186112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11801e0ff0 0x7f1180140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1184e78640 1 --2- 172.21.15.17:0/1943186112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1180145380 0x7f118009eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1185679640 1 -- 172.21.15.17:0/1943186112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11801e0ff0 msgr2=0x7f1180140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1185679640 1 --2- 172.21.15.17:0/1943186112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11801e0ff0 0x7f1180140e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1185679640 1 -- 172.21.15.17:0/1943186112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1180145380 msgr2=0x7f118009eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1185679640 1 --2- 172.21.15.17:0/1943186112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1180145380 0x7f118009eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1185679640 1 -- 172.21.15.17:0/1943186112 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1180141360 con 0x7f1180067220 2024-03-16T10:11:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1185679640 1 --2- 172.21.15.17:0/1943186112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1180067220 0x7f11801404f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f11740ae040 tx=0x7f1174019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1171ffb640 1 -- 172.21.15.17:0/1943186112 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11740b0040 con 0x7f1180067220 2024-03-16T10:11:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1171ffb640 1 -- 172.21.15.17:0/1943186112 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11740b43f0 con 0x7f1180067220 2024-03-16T10:11:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f11878dd640 1 -- 172.21.15.17:0/1943186112 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f118013d4e0 con 0x7f1180067220 2024-03-16T10:11:32.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1171ffb640 1 -- 172.21.15.17:0/1943186112 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11740b47c0 con 0x7f1180067220 2024-03-16T10:11:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f11878dd640 1 -- 172.21.15.17:0/1943186112 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11801418c0 con 0x7f1180067220 2024-03-16T10:11:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f11878dd640 1 -- 172.21.15.17:0/1943186112 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f118006a130 con 0x7f1180067220 2024-03-16T10:11:32.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1171ffb640 1 -- 172.21.15.17:0/1943186112 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f11740b4960 con 0x7f1180067220 2024-03-16T10:11:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1171ffb640 1 --2- 172.21.15.17:0/1943186112 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1164067660 0x7f1164069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1171ffb640 1 -- 172.21.15.17:0/1943186112 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4943+0+0 (secure 0 0 0) 0x7f117411efe0 con 0x7f1180067220 2024-03-16T10:11:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.536+0000 7f1184e78640 1 --2- 172.21.15.17:0/1943186112 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1164067660 0x7f1164069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:32.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.540+0000 7f1184e78640 1 --2- 172.21.15.17:0/1943186112 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1164067660 0x7f1164069b20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f118009e8d0 tx=0x7f117c007d60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:32.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.540+0000 7f1171ffb640 1 -- 172.21.15.17:0/1943186112 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1180141360 con 0x7f1180067220 2024-03-16T10:11:32.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:32.716+0000 7f11878dd640 1 -- 172.21.15.17:0/1943186112 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f1180144ea0 con 0x7f1180067220 2024-03-16T10:11:33.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.328+0000 7f1171ffb640 1 -- 172.21.15.17:0/1943186112 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) ==== 97+0+0 (secure 0 0 0) 0x7f1174016070 con 0x7f1180067220 2024-03-16T10:11:33.334 INFO:teuthology.orchestra.run.smithi017.stderr: removed all blocklist entries 2024-03-16T10:11:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.332+0000 7f11878dd640 1 -- 172.21.15.17:0/1943186112 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1164067660 msgr2=0x7f1164069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.332+0000 7f11878dd640 1 --2- 172.21.15.17:0/1943186112 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1164067660 0x7f1164069b20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f118009e8d0 tx=0x7f117c007d60 comp rx=0 tx=0).stop 2024-03-16T10:11:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.332+0000 7f11878dd640 1 -- 172.21.15.17:0/1943186112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1180067220 msgr2=0x7f11801404f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.332+0000 7f11878dd640 1 --2- 172.21.15.17:0/1943186112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1180067220 0x7f11801404f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f11740ae040 tx=0x7f1174019190 comp rx=0 tx=0).stop 2024-03-16T10:11:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.332+0000 7f11878dd640 1 -- 172.21.15.17:0/1943186112 shutdown_connections 2024-03-16T10:11:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.332+0000 7f11878dd640 1 --2- 172.21.15.17:0/1943186112 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1164067660 0x7f1164069b20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.332+0000 7f11878dd640 1 --2- 172.21.15.17:0/1943186112 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f11801e0ff0 0x7f1180140e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.332+0000 7f11878dd640 1 --2- 172.21.15.17:0/1943186112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1180145380 0x7f118009eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.332+0000 7f11878dd640 1 --2- 172.21.15.17:0/1943186112 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1180067220 0x7f11801404f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.332+0000 7f11878dd640 1 -- 172.21.15.17:0/1943186112 >> 172.21.15.17:0/1943186112 conn(0x7f11800b9380 msgr2=0x7f118009b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:33.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.332+0000 7f11878dd640 1 -- 172.21.15.17:0/1943186112 shutdown_connections 2024-03-16T10:11:33.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.332+0000 7f11878dd640 1 -- 172.21.15.17:0/1943186112 wait complete. 2024-03-16T10:11:33.359 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-16T10:11:33.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.484+0000 7f8a1aaf9640 1 Processor -- start 2024-03-16T10:11:33.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.484+0000 7f8a1aaf9640 1 -- start start 2024-03-16T10:11:33.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.484+0000 7f8a1aaf9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a14067220 0x7f8a140b5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:33.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.484+0000 7f8a1aaf9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a14145380 0x7f8a1413f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:33.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.484+0000 7f8a1aaf9640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a140b4a60 con 0x7f8a141457c0 2024-03-16T10:11:33.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.484+0000 7f8a1aaf9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8a14148970 con 0x7f8a14145380 2024-03-16T10:11:33.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.484+0000 7f8a1aaf9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8a14146360 con 0x7f8a14067220 2024-03-16T10:11:33.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a13fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a14145380 0x7f8a1413f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:33.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a13fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8a14145380 0x7f8a1413f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51086/0 (socket says 172.21.15.17:51086) 2024-03-16T10:11:33.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a13fff640 1 -- 172.21.15.17:0/3234964254 learned_addr learned my addr 172.21.15.17:0/3234964254 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:33.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a18895640 1 --2- 172.21.15.17:0/3234964254 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a14067220 0x7f8a140b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:33.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a13fff640 1 -- 172.21.15.17:0/3234964254 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a14067220 msgr2=0x7f8a140b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:33.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a13fff640 1 --2- 172.21.15.17:0/3234964254 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a14067220 0x7f8a140b5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a13fff640 1 -- 172.21.15.17:0/3234964254 >> v1:172.21.15.174:6789/0 conn(0x7f8a141457c0 legacy=0x7f8a1406d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:33.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a13fff640 1 -- 172.21.15.17:0/3234964254 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a14067600 con 0x7f8a14145380 2024-03-16T10:11:33.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a13fff640 1 --2- 172.21.15.17:0/3234964254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a14145380 0x7f8a1413f140 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009870 tx=0x7f8a0001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc20062a646ce9a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:33.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a137fe640 1 -- 172.21.15.17:0/3234964254 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000b1070 con 0x7f8a14145380 2024-03-16T10:11:33.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a137fe640 1 -- 172.21.15.17:0/3234964254 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a000ac620 con 0x7f8a14145380 2024-03-16T10:11:33.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a137fe640 1 -- 172.21.15.17:0/3234964254 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000ac9f0 con 0x7f8a14145380 2024-03-16T10:11:33.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/3234964254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a14145380 msgr2=0x7f8a1413f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:33.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a1aaf9640 1 --2- 172.21.15.17:0/3234964254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a14145380 0x7f8a1413f140 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009870 tx=0x7f8a0001c4d0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/3234964254 shutdown_connections 2024-03-16T10:11:33.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a1aaf9640 1 --2- 172.21.15.17:0/3234964254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a14145380 0x7f8a1413f140 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a1aaf9640 1 --2- 172.21.15.17:0/3234964254 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a14067220 0x7f8a140b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/3234964254 >> 172.21.15.17:0/3234964254 conn(0x7f8a140b9380 msgr2=0x7f8a140b9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:33.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/3234964254 shutdown_connections 2024-03-16T10:11:33.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.488+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/3234964254 wait complete. 2024-03-16T10:11:33.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a1aaf9640 1 Processor -- start 2024-03-16T10:11:33.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a1aaf9640 1 -- start start 2024-03-16T10:11:33.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a1aaf9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a14067220 0x7f8a141e1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a1aaf9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a14145380 0x7f8a141475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a1aaf9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a141457c0 0x7f8a140b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a1aaf9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1413fa90 con 0x7f8a14145380 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a1aaf9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8a140b4cd0 con 0x7f8a14067220 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a1aaf9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8a141496a0 con 0x7f8a141457c0 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a13fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a14145380 0x7f8a141475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a13fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a14145380 0x7f8a141475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51098/0 (socket says 172.21.15.17:51098) 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a13fff640 1 -- 172.21.15.17:0/2360096285 learned_addr learned my addr 172.21.15.17:0/2360096285 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a18895640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a14067220 0x7f8a141e1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a19096640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a141457c0 0x7f8a140b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a13fff640 1 -- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a141457c0 msgr2=0x7f8a140b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a13fff640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a141457c0 0x7f8a140b17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a13fff640 1 -- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a14067220 msgr2=0x7f8a141e1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a13fff640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a14067220 0x7f8a141e1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a13fff640 1 -- 172.21.15.17:0/2360096285 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1413d460 con 0x7f8a14145380 2024-03-16T10:11:33.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a18895640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a14067220 0x7f8a141e1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:33.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a13fff640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a14145380 0x7f8a141475c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8a000af040 tx=0x7f8a00019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:33.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a117fa640 1 -- 172.21.15.17:0/2360096285 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000bc070 con 0x7f8a14145380 2024-03-16T10:11:33.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a117fa640 1 -- 172.21.15.17:0/2360096285 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a000b7430 con 0x7f8a14145380 2024-03-16T10:11:33.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a117fa640 1 -- 172.21.15.17:0/2360096285 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000b1040 con 0x7f8a14145380 2024-03-16T10:11:33.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/2360096285 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1413f090 con 0x7f8a14145380 2024-03-16T10:11:33.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.492+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/2360096285 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a1413edb0 con 0x7f8a14145380 2024-03-16T10:11:33.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.496+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/2360096285 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a1406a130 con 0x7f8a14145380 2024-03-16T10:11:33.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.496+0000 7f8a117fa640 1 -- 172.21.15.17:0/2360096285 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8a000b63d0 con 0x7f8a14145380 2024-03-16T10:11:33.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.496+0000 7f8a117fa640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f89f8067660 0x7f89f8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:33.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.496+0000 7f8a117fa640 1 -- 172.21.15.17:0/2360096285 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7f8a0011ee70 con 0x7f8a14145380 2024-03-16T10:11:33.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.496+0000 7f8a18895640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f89f8067660 0x7f89f8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:33.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.496+0000 7f8a18895640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f89f8067660 0x7f89f8069b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009ae0 tx=0x7f8a08008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:33.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.500+0000 7f8a117fa640 1 -- 172.21.15.17:0/2360096285 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8a00016150 con 0x7f8a14145380 2024-03-16T10:11:33.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.724+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/2360096285 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8a141432f0 con 0x7f8a14145380 2024-03-16T10:11:33.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.724+0000 7f8a117fa640 1 -- 172.21.15.17:0/2360096285 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+5207 (secure 0 0 0) 0x7f8a000efdb0 con 0x7f8a14145380 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: "auth_dump": [ 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.a", 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5b/VlioLRMxAA8TL1+mZbA/7VcCJxYTFtww==", 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mds.b", 2024-03-16T10:11:33.729 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5b/VlOUAVNxAAGphuN1rjaCdQcWq+gH79Gg==", 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.0", 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD7b/VlNBu6CBAAzHRY4hnKP3VMVq6nCYXDKw==", 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.1", 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD+b/Vl1WUSDRAAUC0iQzl7b7N/4tbm6Vm3+w==", 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.2", 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQABcPVlpZwMFBAA7/Ksf/Of6feuG6+1tI4cag==", 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-16T10:11:33.731 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.3", 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAEcPVl70SBGhAAymTiXwnBJPeopd5TS5vsIg==", 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.4", 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAKcPVlTm5CHxAA32mdxIATfNlJUbppaEYvNg==", 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.732 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.5", 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQANcPVl4phgIxAAjlG3iIismsV4ivhIFSjM1g==", 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.6", 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAQcPVlV0ykKRAAFTAAIf4cgeITuayHHi+B8w==", 2024-03-16T10:11:33.733 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "osd.7", 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQATcPVlkOSNMBAA4zy6wXASGhCKcXPrjEcpZA==", 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow profile osd", 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile osd", 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.0", 2024-03-16T10:11:33.734 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD6b/Vl9GM/ARAAE2ObwC6ro6fuTOmqZ2GNzg==", 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow", 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow r", 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow rw", 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow rwx" 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.admin", 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5b/VlNhtOHBAAhmLb/48tOEATTAsd9GGWLg==", 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": "allow *", 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow *", 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.735 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.736 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-mds", 2024-03-16T10:11:33.736 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAZcPVl4cTEERAAV7CxfyqIW5Z0rQxRoC8sEg==", 2024-03-16T10:11:33.736 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.736 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-mds" 2024-03-16T10:11:33.736 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.736 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-mgr", 2024-03-16T10:11:33.736 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAZcPVla9nEERAApVUYNiSkd57lJqRpssx42g==", 2024-03-16T10:11:33.736 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.736 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-16T10:11:33.736 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.736 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-osd", 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAZcPVl3+nEERAAs8yA7+VpyVe1isVfSHSuUw==", 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-osd" 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-rbd", 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAZcPVl1/nEERAAcb+CtLJZ5si7BBZeCgcv8Q==", 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.737 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAZcPVlcwrFERAAEO1zu5W581Tal3YTyVYNZw==", 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "client.bootstrap-rgw", 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQAZcPVlWxvFERAAlHu03L0wbPoQkpKcxaYMXw==", 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.x", 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5b/VlC7UbMBAAmnD6q7ZwPqoXjZfA+iVK9A==", 2024-03-16T10:11:33.738 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: "entity": "mgr.y", 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: "key": "AQD5b/Vlv4ZiLRAAsPpAwuAQsda6/9P0ltUIPQ==", 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: "caps": { 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": "allow *", 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": "allow profile mgr", 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": "allow *" 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-03-16T10:11:33.739 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-16T10:11:33.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.724+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f89f8067660 msgr2=0x7f89f8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:33.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.724+0000 7f8a1aaf9640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f89f8067660 0x7f89f8069b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8a08009ae0 tx=0x7f8a08008040 comp rx=0 tx=0).stop 2024-03-16T10:11:33.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.724+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/2360096285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a14145380 msgr2=0x7f8a141475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:33.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.724+0000 7f8a1aaf9640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a14145380 0x7f8a141475c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8a000af040 tx=0x7f8a00019270 comp rx=0 tx=0).stop 2024-03-16T10:11:33.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.724+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/2360096285 shutdown_connections 2024-03-16T10:11:33.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.724+0000 7f8a1aaf9640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f89f8067660 0x7f89f8069b20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.724+0000 7f8a1aaf9640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8a141457c0 0x7f8a140b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.724+0000 7f8a1aaf9640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a14145380 0x7f8a141475c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.724+0000 7f8a1aaf9640 1 --2- 172.21.15.17:0/2360096285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a14067220 0x7f8a141e1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.724+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/2360096285 >> 172.21.15.17:0/2360096285 conn(0x7f8a140b9380 msgr2=0x7f8a1409aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:33.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.724+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/2360096285 shutdown_connections 2024-03-16T10:11:33.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.724+0000 7f8a1aaf9640 1 -- 172.21.15.17:0/2360096285 wait complete. 2024-03-16T10:11:33.748 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-16T10:11:33.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a9534b640 1 Processor -- start 2024-03-16T10:11:33.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a9534b640 1 -- start start 2024-03-16T10:11:33.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a9534b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a9006d750 0x7f9a90072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:33.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a9534b640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9a90145660 0x7f9a90145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:33.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a9534b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a9006db30 con 0x7f9a900affc0 2024-03-16T10:11:33.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a9534b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9a90073380 con 0x7f9a9006d750 2024-03-16T10:11:33.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a9534b640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f9a900b4c30 con 0x7f9a90145660 2024-03-16T10:11:33.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a8effd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f9a9006d750 0x7f9a90072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:33.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a8e7fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f9a900affc0 0x7f9a900b0c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36164/0 (socket says 172.21.15.17:36164) 2024-03-16T10:11:33.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a8e7fc640 1 -- 172.21.15.17:0/3829232436 learned_addr learned my addr 172.21.15.17:0/3829232436 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:33.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a8f7fe640 1 --2- 172.21.15.17:0/3829232436 >> v2:172.21.15.174:3300/0 conn(0x7f9a90145660 0x7f9a90145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:33.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a8effd640 1 -- 172.21.15.17:0/3829232436 >> v2:172.21.15.174:3300/0 conn(0x7f9a90145660 msgr2=0x7f9a90145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:33.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a8effd640 1 --2- 172.21.15.17:0/3829232436 >> v2:172.21.15.174:3300/0 conn(0x7f9a90145660 0x7f9a90145ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a8effd640 1 -- 172.21.15.17:0/3829232436 >> v1:172.21.15.17:6789/0 conn(0x7f9a900affc0 legacy=0x7f9a900b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:33.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a8effd640 1 -- 172.21.15.17:0/3829232436 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a900b4f10 con 0x7f9a9006d750 2024-03-16T10:11:33.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.860+0000 7f9a8f7fe640 1 --2- 172.21.15.17:0/3829232436 >> v2:172.21.15.174:3300/0 conn(0x7f9a90145660 0x7f9a90145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:33.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.864+0000 7f9a8effd640 1 --2- 172.21.15.17:0/3829232436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9006d750 0x7f9a90072e40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9a900b26e0 tx=0x7f9a8401cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4be5091a241cd7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:33.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.864+0000 7f9a8dffb640 1 -- 172.21.15.17:0/3829232436 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a840b1070 con 0x7f9a9006d750 2024-03-16T10:11:33.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.864+0000 7f9a8dffb640 1 -- 172.21.15.17:0/3829232436 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a840ac5f0 con 0x7f9a9006d750 2024-03-16T10:11:33.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.864+0000 7f9a8dffb640 1 -- 172.21.15.17:0/3829232436 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a840ac9c0 con 0x7f9a9006d750 2024-03-16T10:11:33.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.864+0000 7f9a9534b640 1 -- 172.21.15.17:0/3829232436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9006d750 msgr2=0x7f9a90072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:33.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.864+0000 7f9a9534b640 1 --2- 172.21.15.17:0/3829232436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9006d750 0x7f9a90072e40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9a900b26e0 tx=0x7f9a8401cee0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.864+0000 7f9a9534b640 1 -- 172.21.15.17:0/3829232436 shutdown_connections 2024-03-16T10:11:33.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.864+0000 7f9a9534b640 1 --2- 172.21.15.17:0/3829232436 >> v2:172.21.15.174:3300/0 conn(0x7f9a90145660 0x7f9a90145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.864+0000 7f9a9534b640 1 --2- 172.21.15.17:0/3829232436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9006d750 0x7f9a90072e40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.864+0000 7f9a9534b640 1 -- 172.21.15.17:0/3829232436 >> 172.21.15.17:0/3829232436 conn(0x7f9a900b55c0 msgr2=0x7f9a900b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:33.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.864+0000 7f9a9534b640 1 -- 172.21.15.17:0/3829232436 shutdown_connections 2024-03-16T10:11:33.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.864+0000 7f9a9534b640 1 -- 172.21.15.17:0/3829232436 wait complete. 2024-03-16T10:11:33.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.864+0000 7f9a9534b640 1 Processor -- start 2024-03-16T10:11:33.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.864+0000 7f9a9534b640 1 -- start start 2024-03-16T10:11:33.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a9534b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9006d750 0x7f9a90144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:33.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a9534b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a900affc0 0x7f9a901e7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:33.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a9534b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a90145660 0x7f9a901e7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:33.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a9534b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9a90158af0 con 0x7f9a9006d750 2024-03-16T10:11:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a9534b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9a90158970 con 0x7f9a90145660 2024-03-16T10:11:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a9534b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9a90158c70 con 0x7f9a900affc0 2024-03-16T10:11:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a8effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9006d750 0x7f9a90144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a8effd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9006d750 0x7f9a90144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51110/0 (socket says 172.21.15.17:51110) 2024-03-16T10:11:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a8effd640 1 -- 172.21.15.17:0/3164890102 learned_addr learned my addr 172.21.15.17:0/3164890102 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a8e7fc640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a900affc0 0x7f9a901e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a8effd640 1 -- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a900affc0 msgr2=0x7f9a901e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a8f7fe640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a90145660 0x7f9a901e7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a8effd640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a900affc0 0x7f9a901e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a8effd640 1 -- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a90145660 msgr2=0x7f9a901e7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a8effd640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a90145660 0x7f9a901e7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a8effd640 1 -- 172.21.15.17:0/3164890102 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a900aa2f0 con 0x7f9a9006d750 2024-03-16T10:11:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a8e7fc640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a900affc0 0x7f9a901e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:33.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a8effd640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9006d750 0x7f9a90144110 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9a840af040 tx=0x7f9a84019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:33.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a777fe640 1 -- 172.21.15.17:0/3164890102 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a840b1040 con 0x7f9a9006d750 2024-03-16T10:11:33.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a777fe640 1 -- 172.21.15.17:0/3164890102 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a840bb070 con 0x7f9a9006d750 2024-03-16T10:11:33.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a9534b640 1 -- 172.21.15.17:0/3164890102 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a90147ca0 con 0x7f9a9006d750 2024-03-16T10:11:33.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a777fe640 1 -- 172.21.15.17:0/3164890102 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a84006e20 con 0x7f9a9006d750 2024-03-16T10:11:33.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a9534b640 1 -- 172.21.15.17:0/3164890102 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a90144e00 con 0x7f9a9006d750 2024-03-16T10:11:33.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a9534b640 1 -- 172.21.15.17:0/3164890102 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a90149600 con 0x7f9a9006d750 2024-03-16T10:11:33.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a777fe640 1 -- 172.21.15.17:0/3164890102 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f9a840b5630 con 0x7f9a9006d750 2024-03-16T10:11:33.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a777fe640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a7c067660 0x7f9a7c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:33.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a777fe640 1 -- 172.21.15.17:0/3164890102 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7f9a8411f220 con 0x7f9a9006d750 2024-03-16T10:11:33.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.868+0000 7f9a8e7fc640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a7c067660 0x7f9a7c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:33.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.872+0000 7f9a8e7fc640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a7c067660 0x7f9a7c069b20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9a90144a00 tx=0x7f9a80009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:33.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:33.872+0000 7f9a777fe640 1 -- 172.21.15.17:0/3164890102 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9a840a9070 con 0x7f9a9006d750 2024-03-16T10:11:34.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.092+0000 7f9a9534b640 1 -- 172.21.15.17:0/3164890102 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f9a90068360 con 0x7f9a9006d750 2024-03-16T10:11:34.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.096+0000 7f9a777fe640 1 -- 172.21.15.17:0/3164890102 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7f9a84016070 con 0x7f9a9006d750 2024-03-16T10:11:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.100+0000 7f9a9534b640 1 -- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a7c067660 msgr2=0x7f9a7c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.100+0000 7f9a9534b640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a7c067660 0x7f9a7c069b20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9a90144a00 tx=0x7f9a80009040 comp rx=0 tx=0).stop 2024-03-16T10:11:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.100+0000 7f9a9534b640 1 -- 172.21.15.17:0/3164890102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9006d750 msgr2=0x7f9a90144110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.100+0000 7f9a9534b640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9006d750 0x7f9a90144110 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9a840af040 tx=0x7f9a84019220 comp rx=0 tx=0).stop 2024-03-16T10:11:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.100+0000 7f9a9534b640 1 -- 172.21.15.17:0/3164890102 shutdown_connections 2024-03-16T10:11:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.100+0000 7f9a9534b640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9a7c067660 0x7f9a7c069b20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.100+0000 7f9a9534b640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a90145660 0x7f9a901e7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.100+0000 7f9a9534b640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9a900affc0 0x7f9a901e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.100+0000 7f9a9534b640 1 --2- 172.21.15.17:0/3164890102 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9a9006d750 0x7f9a90144110 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.100+0000 7f9a9534b640 1 -- 172.21.15.17:0/3164890102 >> 172.21.15.17:0/3164890102 conn(0x7f9a900b55c0 msgr2=0x7f9a9014b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:34.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.100+0000 7f9a9534b640 1 -- 172.21.15.17:0/3164890102 shutdown_connections 2024-03-16T10:11:34.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.100+0000 7f9a9534b640 1 -- 172.21.15.17:0/3164890102 wait complete. 2024-03-16T10:11:34.122 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-16T10:11:34.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd8a24640 1 Processor -- start 2024-03-16T10:11:34.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd8a24640 1 -- start start 2024-03-16T10:11:34.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd8a24640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4cd40b4650 0x7f4cd414b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:34.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd8a24640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cd40b7650 0x7f4cd41412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:34.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd8a24640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4cd4141820 0x7f4cd41474d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:34.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd8a24640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f4cd4141c50 con 0x7f4cd40b4650 2024-03-16T10:11:34.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd8a24640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f4cd40b4a30 con 0x7f4cd4141820 2024-03-16T10:11:34.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd8a24640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd414abe0 con 0x7f4cd40b7650 2024-03-16T10:11:34.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4cd40b4650 0x7f4cd414b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:34.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f4cd40b4650 0x7f4cd414b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51116/0 (socket says 172.21.15.17:51116) 2024-03-16T10:11:34.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd259c640 1 -- 172.21.15.17:0/976457269 learned_addr learned my addr 172.21.15.17:0/976457269 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:34.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd2d9d640 1 --2- 172.21.15.17:0/976457269 >> v2:172.21.15.174:3300/0 conn(0x7f4cd4141820 0x7f4cd41474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:34.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd259c640 1 -- 172.21.15.17:0/976457269 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cd40b7650 msgr2=0x7f4cd41412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:34.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd259c640 1 --2- 172.21.15.17:0/976457269 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cd40b7650 0x7f4cd41412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd259c640 1 -- 172.21.15.17:0/976457269 >> v2:172.21.15.174:3300/0 conn(0x7f4cd4141820 msgr2=0x7f4cd41474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:34.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd259c640 1 --2- 172.21.15.17:0/976457269 >> v2:172.21.15.174:3300/0 conn(0x7f4cd4141820 0x7f4cd41474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd259c640 1 -- 172.21.15.17:0/976457269 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd414aec0 con 0x7f4cd40b4650 2024-03-16T10:11:34.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.244+0000 7f4cd2d9d640 1 --2- 172.21.15.17:0/976457269 >> v2:172.21.15.174:3300/0 conn(0x7f4cd4141820 0x7f4cd41474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd259c640 1 --2- 172.21.15.17:0/976457269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cd40b4650 0x7f4cd414b3e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40b3c30 tx=0x7f4ccc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf791fc6f45be09 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd159a640 1 -- 172.21.15.17:0/976457269 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc0a5690 con 0x7f4cd40b4650 2024-03-16T10:11:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd159a640 1 -- 172.21.15.17:0/976457269 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ccc0a5830 con 0x7f4cd40b4650 2024-03-16T10:11:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd159a640 1 -- 172.21.15.17:0/976457269 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc0a5c00 con 0x7f4cd40b4650 2024-03-16T10:11:34.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd8a24640 1 -- 172.21.15.17:0/976457269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cd40b4650 msgr2=0x7f4cd414b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:34.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd8a24640 1 --2- 172.21.15.17:0/976457269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cd40b4650 0x7f4cd414b3e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40b3c30 tx=0x7f4ccc01c960 comp rx=0 tx=0).stop 2024-03-16T10:11:34.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd8a24640 1 -- 172.21.15.17:0/976457269 shutdown_connections 2024-03-16T10:11:34.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd8a24640 1 --2- 172.21.15.17:0/976457269 >> v2:172.21.15.174:3300/0 conn(0x7f4cd4141820 0x7f4cd41474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd8a24640 1 --2- 172.21.15.17:0/976457269 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cd40b7650 0x7f4cd41412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd8a24640 1 --2- 172.21.15.17:0/976457269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cd40b4650 0x7f4cd414b3e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd8a24640 1 -- 172.21.15.17:0/976457269 >> 172.21.15.17:0/976457269 conn(0x7f4cd40bccd0 msgr2=0x7f4cd40bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:34.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd8a24640 1 -- 172.21.15.17:0/976457269 shutdown_connections 2024-03-16T10:11:34.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd8a24640 1 -- 172.21.15.17:0/976457269 wait complete. 2024-03-16T10:11:34.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd8a24640 1 Processor -- start 2024-03-16T10:11:34.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.248+0000 7f4cd8a24640 1 -- start start 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd8a24640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd40b4650 0x7f4cd4146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd8a24640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cd40b7650 0x7f4cd41e33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd8a24640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cd4141820 0x7f4cd41e3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd8a24640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd40b94a0 con 0x7f4cd4141820 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd8a24640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd40b9320 con 0x7f4cd40b4650 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd8a24640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4cd40b59f0 con 0x7f4cd40b7650 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd2d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cd4141820 0x7f4cd41e3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd2d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cd4141820 0x7f4cd41e3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51118/0 (socket says 172.21.15.17:51118) 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd2d9d640 1 -- 172.21.15.17:0/2247446219 learned_addr learned my addr 172.21.15.17:0/2247446219 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd259c640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd40b4650 0x7f4cd4146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd1d9b640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cd40b7650 0x7f4cd41e33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd2d9d640 1 -- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cd40b7650 msgr2=0x7f4cd41e33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd2d9d640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cd40b7650 0x7f4cd41e33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd2d9d640 1 -- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd40b4650 msgr2=0x7f4cd4146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:34.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd2d9d640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd40b4650 0x7f4cd4146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd2d9d640 1 -- 172.21.15.17:0/2247446219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd4142b90 con 0x7f4cd4141820 2024-03-16T10:11:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd259c640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd40b4650 0x7f4cd4146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd1d9b640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cd40b7650 0x7f4cd41e33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd2d9d640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cd4141820 0x7f4cd41e3910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400c7f0 tx=0x7f4cc400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cc2ffd640 1 -- 172.21.15.17:0/2247446219 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4015020 con 0x7f4cd4141820 2024-03-16T10:11:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cc2ffd640 1 -- 172.21.15.17:0/2247446219 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4cc4004250 con 0x7f4cd4141820 2024-03-16T10:11:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd8a24640 1 -- 172.21.15.17:0/2247446219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd40b3660 con 0x7f4cd4141820 2024-03-16T10:11:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cc2ffd640 1 -- 172.21.15.17:0/2247446219 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4003620 con 0x7f4cd4141820 2024-03-16T10:11:34.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd8a24640 1 -- 172.21.15.17:0/2247446219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd4142e00 con 0x7f4cd4141820 2024-03-16T10:11:34.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd8a24640 1 -- 172.21.15.17:0/2247446219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cd4068360 con 0x7f4cd4141820 2024-03-16T10:11:34.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cc2ffd640 1 -- 172.21.15.17:0/2247446219 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f4cc4018020 con 0x7f4cd4141820 2024-03-16T10:11:34.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cc2ffd640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4cb4067660 0x7f4cb4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:34.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cc2ffd640 1 -- 172.21.15.17:0/2247446219 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7f4cc40857d0 con 0x7f4cd4141820 2024-03-16T10:11:34.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.252+0000 7f4cd259c640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4cb4067660 0x7f4cb4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:34.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.256+0000 7f4cd259c640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4cb4067660 0x7f4cb4069b20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0ae000 tx=0x7f4ccc0a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:34.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.256+0000 7f4cc2ffd640 1 -- 172.21.15.17:0/2247446219 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4cc401f020 con 0x7f4cd4141820 2024-03-16T10:11:34.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.476+0000 7f4cd8a24640 1 -- 172.21.15.17:0/2247446219 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f4cd40b90c0 con 0x7f4cd4141820 2024-03-16T10:11:34.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.480+0000 7f4cc2ffd640 1 -- 172.21.15.17:0/2247446219 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f4cc4056790 con 0x7f4cd4141820 2024-03-16T10:11:34.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.484+0000 7f4cd8a24640 1 -- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4cb4067660 msgr2=0x7f4cb4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:34.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.484+0000 7f4cd8a24640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4cb4067660 0x7f4cb4069b20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0ae000 tx=0x7f4ccc0a7070 comp rx=0 tx=0).stop 2024-03-16T10:11:34.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.484+0000 7f4cd8a24640 1 -- 172.21.15.17:0/2247446219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cd4141820 msgr2=0x7f4cd41e3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:34.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.484+0000 7f4cd8a24640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cd4141820 0x7f4cd41e3910 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400c7f0 tx=0x7f4cc400ccb0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.484+0000 7f4cd8a24640 1 -- 172.21.15.17:0/2247446219 shutdown_connections 2024-03-16T10:11:34.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.484+0000 7f4cd8a24640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4cb4067660 0x7f4cb4069b20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.484+0000 7f4cd8a24640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4cd4141820 0x7f4cd41e3910 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.484+0000 7f4cd8a24640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4cd40b7650 0x7f4cd41e33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.484+0000 7f4cd8a24640 1 --2- 172.21.15.17:0/2247446219 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cd40b4650 0x7f4cd4146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.484+0000 7f4cd8a24640 1 -- 172.21.15.17:0/2247446219 >> 172.21.15.17:0/2247446219 conn(0x7f4cd40bccd0 msgr2=0x7f4cd40b7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:34.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.484+0000 7f4cd8a24640 1 -- 172.21.15.17:0/2247446219 shutdown_connections 2024-03-16T10:11:34.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.484+0000 7f4cd8a24640 1 -- 172.21.15.17:0/2247446219 wait complete. 2024-03-16T10:11:34.507 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-16T10:11:34.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62b5010640 1 Processor -- start 2024-03-16T10:11:34.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62b5010640 1 -- start start 2024-03-16T10:11:34.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62b5010640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f62b00b4650 0x7f62b014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:34.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62b5010640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f62b00b7650 0x7f62b01412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:34.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62b5010640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62b0141c30 con 0x7f62b0141820 2024-03-16T10:11:34.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62b5010640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f62b00b4a30 con 0x7f62b00b7650 2024-03-16T10:11:34.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62b5010640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f62b014abe0 con 0x7f62b00b4650 2024-03-16T10:11:34.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62ae59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f62b00b7650 0x7f62b01412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:34.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62ae59c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f62b00b7650 0x7f62b01412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51128/0 (socket says 172.21.15.17:51128) 2024-03-16T10:11:34.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62ae59c640 1 -- 172.21.15.17:0/3973766084 learned_addr learned my addr 172.21.15.17:0/3973766084 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:34.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62aed9d640 1 --2- 172.21.15.17:0/3973766084 >> v2:172.21.15.174:3300/0 conn(0x7f62b00b4650 0x7f62b014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:34.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62ae59c640 1 -- 172.21.15.17:0/3973766084 >> v2:172.21.15.174:3300/0 conn(0x7f62b00b4650 msgr2=0x7f62b014b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:34.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62ae59c640 1 --2- 172.21.15.17:0/3973766084 >> v2:172.21.15.174:3300/0 conn(0x7f62b00b4650 0x7f62b014b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62ae59c640 1 -- 172.21.15.17:0/3973766084 >> v1:172.21.15.17:6789/0 conn(0x7f62b0141820 legacy=0x7f62b01474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:34.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.628+0000 7f62ae59c640 1 -- 172.21.15.17:0/3973766084 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b014aec0 con 0x7f62b00b7650 2024-03-16T10:11:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62ae59c640 1 --2- 172.21.15.17:0/3973766084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62b00b7650 0x7f62b01412e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f62b00ba870 tx=0x7f629801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10520fa7b5bb8f4f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62add9b640 1 -- 172.21.15.17:0/3973766084 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62980b0070 con 0x7f62b00b7650 2024-03-16T10:11:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62add9b640 1 -- 172.21.15.17:0/3973766084 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62980ab600 con 0x7f62b00b7650 2024-03-16T10:11:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62add9b640 1 -- 172.21.15.17:0/3973766084 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62980ab9d0 con 0x7f62b00b7650 2024-03-16T10:11:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 -- 172.21.15.17:0/3973766084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62b00b7650 msgr2=0x7f62b01412e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 --2- 172.21.15.17:0/3973766084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62b00b7650 0x7f62b01412e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f62b00ba870 tx=0x7f629801c490 comp rx=0 tx=0).stop 2024-03-16T10:11:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 -- 172.21.15.17:0/3973766084 shutdown_connections 2024-03-16T10:11:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 --2- 172.21.15.17:0/3973766084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62b00b7650 0x7f62b01412e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 --2- 172.21.15.17:0/3973766084 >> v2:172.21.15.174:3300/0 conn(0x7f62b00b4650 0x7f62b014b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 -- 172.21.15.17:0/3973766084 >> 172.21.15.17:0/3973766084 conn(0x7f62b00bccd0 msgr2=0x7f62b00bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 -- 172.21.15.17:0/3973766084 shutdown_connections 2024-03-16T10:11:34.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 -- 172.21.15.17:0/3973766084 wait complete. 2024-03-16T10:11:34.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 Processor -- start 2024-03-16T10:11:34.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 -- start start 2024-03-16T10:11:34.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62b00b4650 0x7f62b0136420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:34.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b00b7650 0x7f62b0149b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:34.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62b0141820 0x7f62b014a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:34.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f62b00b94a0 con 0x7f62b00b4650 2024-03-16T10:11:34.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62b00b9320 con 0x7f62b00b7650 2024-03-16T10:11:34.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62b5010640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f62b00b59f0 con 0x7f62b0141820 2024-03-16T10:11:34.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62aed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62b00b4650 0x7f62b0136420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:34.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62aed9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62b00b4650 0x7f62b0136420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51132/0 (socket says 172.21.15.17:51132) 2024-03-16T10:11:34.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62aed9d640 1 -- 172.21.15.17:0/4212717615 learned_addr learned my addr 172.21.15.17:0/4212717615 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:34.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62aed9d640 1 -- 172.21.15.17:0/4212717615 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62b0141820 msgr2=0x7f62b014a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:34.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62aed9d640 1 --2- 172.21.15.17:0/4212717615 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62b0141820 0x7f62b014a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62aed9d640 1 -- 172.21.15.17:0/4212717615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b00b7650 msgr2=0x7f62b0149b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:34.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62aed9d640 1 --2- 172.21.15.17:0/4212717615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b00b7650 0x7f62b0149b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62aed9d640 1 -- 172.21.15.17:0/4212717615 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b0142710 con 0x7f62b00b4650 2024-03-16T10:11:34.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62aed9d640 1 --2- 172.21.15.17:0/4212717615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62b00b4650 0x7f62b0136420 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f62a8007ef0 tx=0x7f62a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:34.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62a77fe640 1 -- 172.21.15.17:0/4212717615 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8014620 con 0x7f62b00b4650 2024-03-16T10:11:34.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62a77fe640 1 -- 172.21.15.17:0/4212717615 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a8019070 con 0x7f62b00b4650 2024-03-16T10:11:34.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.632+0000 7f62a77fe640 1 -- 172.21.15.17:0/4212717615 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8014a00 con 0x7f62b00b4650 2024-03-16T10:11:34.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.636+0000 7f62b5010640 1 -- 172.21.15.17:0/4212717615 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b0148f40 con 0x7f62b00b4650 2024-03-16T10:11:34.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.636+0000 7f62b5010640 1 -- 172.21.15.17:0/4212717615 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b0139420 con 0x7f62b00b4650 2024-03-16T10:11:34.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.636+0000 7f62b5010640 1 -- 172.21.15.17:0/4212717615 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b0068360 con 0x7f62b00b4650 2024-03-16T10:11:34.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.636+0000 7f62a77fe640 1 -- 172.21.15.17:0/4212717615 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f62a8014ba0 con 0x7f62b00b4650 2024-03-16T10:11:34.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.636+0000 7f62a77fe640 1 --2- 172.21.15.17:0/4212717615 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6290067690 0x7f6290069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:34.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.636+0000 7f62a77fe640 1 -- 172.21.15.17:0/4212717615 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7f62a8085de0 con 0x7f62b00b4650 2024-03-16T10:11:34.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.636+0000 7f62ae59c640 1 --2- 172.21.15.17:0/4212717615 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6290067690 0x7f6290069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:34.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.636+0000 7f62ae59c640 1 --2- 172.21.15.17:0/4212717615 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6290067690 0x7f6290069b50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f62b006d9b0 tx=0x7f6298004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:34.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.640+0000 7f62a77fe640 1 -- 172.21.15.17:0/4212717615 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f62a8056e40 con 0x7f62b00b4650 2024-03-16T10:11:34.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.864+0000 7f62b5010640 1 -- 172.21.15.17:0/4212717615 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f62b00baad0 con 0x7f62b00b4650 2024-03-16T10:11:34.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.868+0000 7f62a77fe640 1 -- 172.21.15.17:0/4212717615 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f62a8056740 con 0x7f62b00b4650 2024-03-16T10:11:34.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.868+0000 7f62b5010640 1 -- 172.21.15.17:0/4212717615 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6290067690 msgr2=0x7f6290069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:34.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.868+0000 7f62b5010640 1 --2- 172.21.15.17:0/4212717615 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6290067690 0x7f6290069b50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f62b006d9b0 tx=0x7f6298004740 comp rx=0 tx=0).stop 2024-03-16T10:11:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.868+0000 7f62b5010640 1 -- 172.21.15.17:0/4212717615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62b00b4650 msgr2=0x7f62b0136420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.868+0000 7f62b5010640 1 --2- 172.21.15.17:0/4212717615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62b00b4650 0x7f62b0136420 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f62a8007ef0 tx=0x7f62a800e4b0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.868+0000 7f62b5010640 1 -- 172.21.15.17:0/4212717615 shutdown_connections 2024-03-16T10:11:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.868+0000 7f62b5010640 1 --2- 172.21.15.17:0/4212717615 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6290067690 0x7f6290069b50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.868+0000 7f62b5010640 1 --2- 172.21.15.17:0/4212717615 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f62b0141820 0x7f62b014a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.868+0000 7f62b5010640 1 --2- 172.21.15.17:0/4212717615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b00b7650 0x7f62b0149b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.868+0000 7f62b5010640 1 --2- 172.21.15.17:0/4212717615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f62b00b4650 0x7f62b0136420 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.868+0000 7f62b5010640 1 -- 172.21.15.17:0/4212717615 >> 172.21.15.17:0/4212717615 conn(0x7f62b00bccd0 msgr2=0x7f62b00bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.868+0000 7f62b5010640 1 -- 172.21.15.17:0/4212717615 shutdown_connections 2024-03-16T10:11:34.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:34.868+0000 7f62b5010640 1 -- 172.21.15.17:0/4212717615 wait complete. 2024-03-16T10:11:34.892 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-16T10:11:35.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8f68dc640 1 Processor -- start 2024-03-16T10:11:35.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8f68dc640 1 -- start start 2024-03-16T10:11:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8f68dc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8f00b7360 0x7fd8f006f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8f68dc640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fd8f0145720 0x7fd8f0145b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8f68dc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8f01460d0 con 0x7fd8f006e930 2024-03-16T10:11:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8f68dc640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd8f006ed10 con 0x7fd8f00b7360 2024-03-16T10:11:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8f68dc640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fd8f006f850 con 0x7fd8f0145720 2024-03-16T10:11:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8e7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8f00b7360 0x7fd8f006f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8effff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd8f006e930 0x7fd8f013f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36180/0 (socket says 172.21.15.17:36180) 2024-03-16T10:11:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8effff640 1 -- 172.21.15.17:0/160855606 learned_addr learned my addr 172.21.15.17:0/160855606 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8f4e79640 1 --2- 172.21.15.17:0/160855606 >> v2:172.21.15.174:3300/0 conn(0x7fd8f0145720 0x7fd8f0145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8e7fff640 1 -- 172.21.15.17:0/160855606 >> v2:172.21.15.174:3300/0 conn(0x7fd8f0145720 msgr2=0x7fd8f0145b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8e7fff640 1 --2- 172.21.15.17:0/160855606 >> v2:172.21.15.174:3300/0 conn(0x7fd8f0145720 0x7fd8f0145b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8e7fff640 1 -- 172.21.15.17:0/160855606 >> v1:172.21.15.17:6789/0 conn(0x7fd8f006e930 legacy=0x7fd8f013f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.020+0000 7fd8e7fff640 1 -- 172.21.15.17:0/160855606 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f006fb30 con 0x7fd8f00b7360 2024-03-16T10:11:35.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8e7fff640 1 --2- 172.21.15.17:0/160855606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8f00b7360 0x7fd8f006f230 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00715c0 tx=0x7fd8d801c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91eae8ca63094b41 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:35.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8ef7fe640 1 -- 172.21.15.17:0/160855606 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d80b0070 con 0x7fd8f00b7360 2024-03-16T10:11:35.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8ef7fe640 1 -- 172.21.15.17:0/160855606 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8d80ab600 con 0x7fd8f00b7360 2024-03-16T10:11:35.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8ef7fe640 1 -- 172.21.15.17:0/160855606 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d80ab9d0 con 0x7fd8f00b7360 2024-03-16T10:11:35.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 -- 172.21.15.17:0/160855606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8f00b7360 msgr2=0x7fd8f006f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 --2- 172.21.15.17:0/160855606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8f00b7360 0x7fd8f006f230 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00715c0 tx=0x7fd8d801c280 comp rx=0 tx=0).stop 2024-03-16T10:11:35.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 -- 172.21.15.17:0/160855606 shutdown_connections 2024-03-16T10:11:35.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 --2- 172.21.15.17:0/160855606 >> v2:172.21.15.174:3300/0 conn(0x7fd8f0145720 0x7fd8f0145b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 --2- 172.21.15.17:0/160855606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8f00b7360 0x7fd8f006f230 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 -- 172.21.15.17:0/160855606 >> 172.21.15.17:0/160855606 conn(0x7fd8f0071910 msgr2=0x7fd8f0071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:35.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 -- 172.21.15.17:0/160855606 shutdown_connections 2024-03-16T10:11:35.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 -- 172.21.15.17:0/160855606 wait complete. 2024-03-16T10:11:35.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 Processor -- start 2024-03-16T10:11:35.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 -- start start 2024-03-16T10:11:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8f006e930 0x7fd8f0142700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8f00b7360 0x7fd8f013a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f0145720 0x7fd8f013fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f0152720 con 0x7fd8f00b7360 2024-03-16T10:11:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f01525a0 con 0x7fd8f0145720 2024-03-16T10:11:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f68dc640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f01528a0 con 0x7fd8f006e930 2024-03-16T10:11:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8f00b7360 0x7fd8f013a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8e7fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8f00b7360 0x7fd8f013a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51150/0 (socket says 172.21.15.17:51150) 2024-03-16T10:11:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8e7fff640 1 -- 172.21.15.17:0/2669111731 learned_addr learned my addr 172.21.15.17:0/2669111731 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8effff640 1 --2- 172.21.15.17:0/2669111731 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8f006e930 0x7fd8f0142700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8f4e79640 1 --2- 172.21.15.17:0/2669111731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f0145720 0x7fd8f013fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8e7fff640 1 -- 172.21.15.17:0/2669111731 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8f006e930 msgr2=0x7fd8f0142700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8e7fff640 1 --2- 172.21.15.17:0/2669111731 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8f006e930 0x7fd8f0142700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8e7fff640 1 -- 172.21.15.17:0/2669111731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f0145720 msgr2=0x7fd8f013fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8e7fff640 1 --2- 172.21.15.17:0/2669111731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f0145720 0x7fd8f013fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8e7fff640 1 -- 172.21.15.17:0/2669111731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f01401f0 con 0x7fd8f00b7360 2024-03-16T10:11:35.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.024+0000 7fd8e7fff640 1 --2- 172.21.15.17:0/2669111731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8f00b7360 0x7fd8f013a240 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80ae040 tx=0x7fd8d8019020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:35.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.028+0000 7fd8ed7fa640 1 -- 172.21.15.17:0/2669111731 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d80b8020 con 0x7fd8f00b7360 2024-03-16T10:11:35.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.028+0000 7fd8ed7fa640 1 -- 172.21.15.17:0/2669111731 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8d80b4630 con 0x7fd8f00b7360 2024-03-16T10:11:35.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.028+0000 7fd8ed7fa640 1 -- 172.21.15.17:0/2669111731 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d80b0040 con 0x7fd8f00b7360 2024-03-16T10:11:35.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.028+0000 7fd8f68dc640 1 -- 172.21.15.17:0/2669111731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f00b2140 con 0x7fd8f00b7360 2024-03-16T10:11:35.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.028+0000 7fd8f68dc640 1 -- 172.21.15.17:0/2669111731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f0141f40 con 0x7fd8f00b7360 2024-03-16T10:11:35.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.028+0000 7fd8f68dc640 1 -- 172.21.15.17:0/2669111731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f0068360 con 0x7fd8f00b7360 2024-03-16T10:11:35.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.028+0000 7fd8ed7fa640 1 -- 172.21.15.17:0/2669111731 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd8d80bb020 con 0x7fd8f00b7360 2024-03-16T10:11:35.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.028+0000 7fd8ed7fa640 1 --2- 172.21.15.17:0/2669111731 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd8d0067660 0x7fd8d0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.028+0000 7fd8ed7fa640 1 -- 172.21.15.17:0/2669111731 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7fd8d811f1f0 con 0x7fd8f00b7360 2024-03-16T10:11:35.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.028+0000 7fd8effff640 1 --2- 172.21.15.17:0/2669111731 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd8d0067660 0x7fd8d0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.028+0000 7fd8effff640 1 --2- 172.21.15.17:0/2669111731 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd8d0067660 0x7fd8d0069b20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0000f70 tx=0x7fd8e0007c90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:35.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.032+0000 7fd8ed7fa640 1 -- 172.21.15.17:0/2669111731 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd8d80c2030 con 0x7fd8f00b7360 2024-03-16T10:11:35.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.248+0000 7fd8f68dc640 1 -- 172.21.15.17:0/2669111731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fd8f00baad0 con 0x7fd8f00b7360 2024-03-16T10:11:35.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.252+0000 7fd8ed7fa640 1 -- 172.21.15.17:0/2669111731 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fd8d8015f90 con 0x7fd8f00b7360 2024-03-16T10:11:35.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.252+0000 7fd8f68dc640 1 -- 172.21.15.17:0/2669111731 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd8d0067660 msgr2=0x7fd8d0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.252+0000 7fd8f68dc640 1 --2- 172.21.15.17:0/2669111731 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd8d0067660 0x7fd8d0069b20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0000f70 tx=0x7fd8e0007c90 comp rx=0 tx=0).stop 2024-03-16T10:11:35.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.252+0000 7fd8f68dc640 1 -- 172.21.15.17:0/2669111731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8f00b7360 msgr2=0x7fd8f013a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.252+0000 7fd8f68dc640 1 --2- 172.21.15.17:0/2669111731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8f00b7360 0x7fd8f013a240 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80ae040 tx=0x7fd8d8019020 comp rx=0 tx=0).stop 2024-03-16T10:11:35.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.252+0000 7fd8f68dc640 1 -- 172.21.15.17:0/2669111731 shutdown_connections 2024-03-16T10:11:35.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.252+0000 7fd8f68dc640 1 --2- 172.21.15.17:0/2669111731 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd8d0067660 0x7fd8d0069b20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.252+0000 7fd8f68dc640 1 --2- 172.21.15.17:0/2669111731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f0145720 0x7fd8f013fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.252+0000 7fd8f68dc640 1 --2- 172.21.15.17:0/2669111731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8f00b7360 0x7fd8f013a240 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.252+0000 7fd8f68dc640 1 --2- 172.21.15.17:0/2669111731 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8f006e930 0x7fd8f0142700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.252+0000 7fd8f68dc640 1 -- 172.21.15.17:0/2669111731 >> 172.21.15.17:0/2669111731 conn(0x7fd8f0071910 msgr2=0x7fd8f0143610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:35.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.252+0000 7fd8f68dc640 1 -- 172.21.15.17:0/2669111731 shutdown_connections 2024-03-16T10:11:35.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.252+0000 7fd8f68dc640 1 -- 172.21.15.17:0/2669111731 wait complete. 2024-03-16T10:11:35.275 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-16T10:11:35.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.384+0000 7f10d3239640 1 Processor -- start 2024-03-16T10:11:35.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.384+0000 7f10d3239640 1 -- start start 2024-03-16T10:11:35.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.384+0000 7f10d3239640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10cc06d750 0x7f10cc06f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.384+0000 7f10d3239640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc06db30 con 0x7f10cc0b2820 2024-03-16T10:11:35.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.384+0000 7f10d3239640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc0b2d30 con 0x7f10cc145280 2024-03-16T10:11:35.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.384+0000 7f10d3239640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f10cc06fcf0 con 0x7f10cc06d750 2024-03-16T10:11:35.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.384+0000 7f10d0fd5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f10cc06d750 0x7f10cc06f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.384+0000 7f10cbfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f10cc0b2820 0x7f10cc13eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36188/0 (socket says 172.21.15.17:36188) 2024-03-16T10:11:35.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.384+0000 7f10cbfff640 1 -- 172.21.15.17:0/766256778 learned_addr learned my addr 172.21.15.17:0/766256778 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:35.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.384+0000 7f10cb7fe640 1 -- 172.21.15.17:0/766256778 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3219304025 0 0) 0x7f10cc06db30 con 0x7f10cc0b2820 2024-03-16T10:11:35.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.384+0000 7f10cb7fe640 1 -- 172.21.15.17:0/766256778 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10b8003530 con 0x7f10cc0b2820 2024-03-16T10:11:35.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.384+0000 7f10d0fd5640 1 -- 172.21.15.17:0/766256778 >> v1:172.21.15.174:6789/0 conn(0x7f10cc145280 legacy=0x7f10cc14fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:35.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.384+0000 7f10d0fd5640 1 -- 172.21.15.17:0/766256778 >> v1:172.21.15.17:6789/0 conn(0x7f10cc0b2820 legacy=0x7f10cc13eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.384+0000 7f10d0fd5640 1 -- 172.21.15.17:0/766256778 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc0afaf0 con 0x7f10cc06d750 2024-03-16T10:11:35.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.388+0000 7f10d0fd5640 1 --2- 172.21.15.17:0/766256778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10cc06d750 0x7f10cc06f7b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f10c40097c0 tx=0x7f10c401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff217df9e35fcf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:35.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.388+0000 7f10cb7fe640 1 -- 172.21.15.17:0/766256778 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c40b0070 con 0x7f10cc06d750 2024-03-16T10:11:35.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.388+0000 7f10cb7fe640 1 -- 172.21.15.17:0/766256778 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c40ab600 con 0x7f10cc06d750 2024-03-16T10:11:35.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.388+0000 7f10cb7fe640 1 -- 172.21.15.17:0/766256778 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c40ab9d0 con 0x7f10cc06d750 2024-03-16T10:11:35.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.388+0000 7f10d3239640 1 -- 172.21.15.17:0/766256778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10cc06d750 msgr2=0x7f10cc06f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.388+0000 7f10d3239640 1 --2- 172.21.15.17:0/766256778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10cc06d750 0x7f10cc06f7b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f10c40097c0 tx=0x7f10c401c340 comp rx=0 tx=0).stop 2024-03-16T10:11:35.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.388+0000 7f10d3239640 1 -- 172.21.15.17:0/766256778 shutdown_connections 2024-03-16T10:11:35.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.388+0000 7f10d3239640 1 --2- 172.21.15.17:0/766256778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10cc06d750 0x7f10cc06f7b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.388+0000 7f10d3239640 1 -- 172.21.15.17:0/766256778 >> 172.21.15.17:0/766256778 conn(0x7f10cc0b55c0 msgr2=0x7f10cc0b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:35.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.388+0000 7f10d3239640 1 -- 172.21.15.17:0/766256778 shutdown_connections 2024-03-16T10:11:35.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.388+0000 7f10d3239640 1 -- 172.21.15.17:0/766256778 wait complete. 2024-03-16T10:11:35.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.388+0000 7f10d3239640 1 Processor -- start 2024-03-16T10:11:35.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.388+0000 7f10d3239640 1 -- start start 2024-03-16T10:11:35.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d3239640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc06d750 0x7f10cc1e3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d3239640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc0b2820 0x7f10cc1e3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d3239640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10cc145280 0x7f10cc1365e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d3239640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc13fa00 con 0x7f10cc145280 2024-03-16T10:11:35.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d3239640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0b49f0 con 0x7f10cc0b2820 2024-03-16T10:11:35.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d3239640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc153c70 con 0x7f10cc06d750 2024-03-16T10:11:35.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d17d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10cc145280 0x7f10cc1365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d17d6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10cc145280 0x7f10cc1365e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51172/0 (socket says 172.21.15.17:51172) 2024-03-16T10:11:35.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d17d6640 1 -- 172.21.15.17:0/147671349 learned_addr learned my addr 172.21.15.17:0/147671349 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:35.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d0fd5640 1 --2- 172.21.15.17:0/147671349 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc06d750 0x7f10cc1e3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10cbfff640 1 --2- 172.21.15.17:0/147671349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc0b2820 0x7f10cc1e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d17d6640 1 -- 172.21.15.17:0/147671349 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc06d750 msgr2=0x7f10cc1e3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d17d6640 1 --2- 172.21.15.17:0/147671349 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc06d750 0x7f10cc1e3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d17d6640 1 -- 172.21.15.17:0/147671349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc0b2820 msgr2=0x7f10cc1e3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d17d6640 1 --2- 172.21.15.17:0/147671349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc0b2820 0x7f10cc1e3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d17d6640 1 -- 172.21.15.17:0/147671349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc135e70 con 0x7f10cc145280 2024-03-16T10:11:35.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d17d6640 1 --2- 172.21.15.17:0/147671349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10cc145280 0x7f10cc1365e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00b9a0 tx=0x7f10bc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:35.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10c97fa640 1 -- 172.21.15.17:0/147671349 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc019070 con 0x7f10cc145280 2024-03-16T10:11:35.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10c97fa640 1 -- 172.21.15.17:0/147671349 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10bc00cba0 con 0x7f10cc145280 2024-03-16T10:11:35.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10c97fa640 1 -- 172.21.15.17:0/147671349 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc014440 con 0x7f10cc145280 2024-03-16T10:11:35.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d3239640 1 -- 172.21.15.17:0/147671349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc134cb0 con 0x7f10cc145280 2024-03-16T10:11:35.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d3239640 1 -- 172.21.15.17:0/147671349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc1351f0 con 0x7f10cc145280 2024-03-16T10:11:35.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d3239640 1 -- 172.21.15.17:0/147671349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10cc068360 con 0x7f10cc145280 2024-03-16T10:11:35.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10c97fa640 1 -- 172.21.15.17:0/147671349 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f10bc01c070 con 0x7f10cc145280 2024-03-16T10:11:35.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10c97fa640 1 --2- 172.21.15.17:0/147671349 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f10ac067660 0x7f10ac069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10c97fa640 1 -- 172.21.15.17:0/147671349 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7f10bc085150 con 0x7f10cc145280 2024-03-16T10:11:35.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.392+0000 7f10d0fd5640 1 --2- 172.21.15.17:0/147671349 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f10ac067660 0x7f10ac069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.396+0000 7f10d0fd5640 1 --2- 172.21.15.17:0/147671349 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f10ac067660 0x7f10ac069b20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f10c40ad000 tx=0x7f10c40a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:35.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.396+0000 7f10c97fa640 1 -- 172.21.15.17:0/147671349 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f10bc0561b0 con 0x7f10cc145280 2024-03-16T10:11:35.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.616+0000 7f10d3239640 1 -- 172.21.15.17:0/147671349 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f10cc0b6bd0 con 0x7f10cc145280 2024-03-16T10:11:35.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.620+0000 7f10c97fa640 1 -- 172.21.15.17:0/147671349 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f10bc055ab0 con 0x7f10cc145280 2024-03-16T10:11:35.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.620+0000 7f10d3239640 1 -- 172.21.15.17:0/147671349 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f10ac067660 msgr2=0x7f10ac069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.620+0000 7f10d3239640 1 --2- 172.21.15.17:0/147671349 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f10ac067660 0x7f10ac069b20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f10c40ad000 tx=0x7f10c40a6000 comp rx=0 tx=0).stop 2024-03-16T10:11:35.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.620+0000 7f10d3239640 1 -- 172.21.15.17:0/147671349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10cc145280 msgr2=0x7f10cc1365e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.620+0000 7f10d3239640 1 --2- 172.21.15.17:0/147671349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10cc145280 0x7f10cc1365e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f10bc00b9a0 tx=0x7f10bc00be60 comp rx=0 tx=0).stop 2024-03-16T10:11:35.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.620+0000 7f10d3239640 1 -- 172.21.15.17:0/147671349 shutdown_connections 2024-03-16T10:11:35.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.620+0000 7f10d3239640 1 --2- 172.21.15.17:0/147671349 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f10ac067660 0x7f10ac069b20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.620+0000 7f10d3239640 1 --2- 172.21.15.17:0/147671349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10cc145280 0x7f10cc1365e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.620+0000 7f10d3239640 1 --2- 172.21.15.17:0/147671349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc0b2820 0x7f10cc1e3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.620+0000 7f10d3239640 1 --2- 172.21.15.17:0/147671349 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f10cc06d750 0x7f10cc1e3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.620+0000 7f10d3239640 1 -- 172.21.15.17:0/147671349 >> 172.21.15.17:0/147671349 conn(0x7f10cc0b55c0 msgr2=0x7f10cc0b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:35.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.620+0000 7f10d3239640 1 -- 172.21.15.17:0/147671349 shutdown_connections 2024-03-16T10:11:35.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.620+0000 7f10d3239640 1 -- 172.21.15.17:0/147671349 wait complete. 2024-03-16T10:11:35.645 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-16T10:11:35.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f9891262640 1 Processor -- start 2024-03-16T10:11:35.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f9891262640 1 -- start start 2024-03-16T10:11:35.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f9891262640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f988c0b2530 0x7f988c0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f9891262640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f988c06d4f0 0x7f988c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f9891262640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f988c143940 con 0x7f988c0b2080 2024-03-16T10:11:35.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f9891262640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f988c0b2b90 con 0x7f988c06d4f0 2024-03-16T10:11:35.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f9891262640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f988c143c00 con 0x7f988c0b2530 2024-03-16T10:11:35.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f988b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f988c06d4f0 0x7f988c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f988affd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f988c0b2080 0x7f988c0affc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36196/0 (socket says 172.21.15.17:36196) 2024-03-16T10:11:35.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f988affd640 1 -- 172.21.15.17:0/1039975238 learned_addr learned my addr 172.21.15.17:0/1039975238 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:35.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f988a7fc640 1 --2- 172.21.15.17:0/1039975238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f988c0b2530 0x7f988c0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f988b7fe640 1 -- 172.21.15.17:0/1039975238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f988c0b2530 msgr2=0x7f988c0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:35.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f988b7fe640 1 --2- 172.21.15.17:0/1039975238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f988c0b2530 0x7f988c0b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f988b7fe640 1 -- 172.21.15.17:0/1039975238 >> v1:172.21.15.17:6789/0 conn(0x7f988c0b2080 legacy=0x7f988c0affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f988b7fe640 1 -- 172.21.15.17:0/1039975238 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988c143ee0 con 0x7f988c06d4f0 2024-03-16T10:11:35.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.764+0000 7f988a7fc640 1 --2- 172.21.15.17:0/1039975238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f988c0b2530 0x7f988c0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:35.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.768+0000 7f988b7fe640 1 --2- 172.21.15.17:0/1039975238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f988c06d4f0 0x7f988c071bb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f98840097c0 tx=0x7f988401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1741a53350bdecc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:35.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.768+0000 7f9889ffb640 1 -- 172.21.15.17:0/1039975238 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98840b0070 con 0x7f988c06d4f0 2024-03-16T10:11:35.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.768+0000 7f9889ffb640 1 -- 172.21.15.17:0/1039975238 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98840ab600 con 0x7f988c06d4f0 2024-03-16T10:11:35.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.768+0000 7f9889ffb640 1 -- 172.21.15.17:0/1039975238 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98840ab9d0 con 0x7f988c06d4f0 2024-03-16T10:11:35.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.768+0000 7f9891262640 1 -- 172.21.15.17:0/1039975238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f988c06d4f0 msgr2=0x7f988c071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.768+0000 7f9891262640 1 --2- 172.21.15.17:0/1039975238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f988c06d4f0 0x7f988c071bb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f98840097c0 tx=0x7f988401c410 comp rx=0 tx=0).stop 2024-03-16T10:11:35.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.768+0000 7f9891262640 1 -- 172.21.15.17:0/1039975238 shutdown_connections 2024-03-16T10:11:35.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.768+0000 7f9891262640 1 --2- 172.21.15.17:0/1039975238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f988c06d4f0 0x7f988c071bb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.768+0000 7f9891262640 1 --2- 172.21.15.17:0/1039975238 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f988c0b2530 0x7f988c0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.768+0000 7f9891262640 1 -- 172.21.15.17:0/1039975238 >> 172.21.15.17:0/1039975238 conn(0x7f988c0b55c0 msgr2=0x7f988c0b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.768+0000 7f9891262640 1 -- 172.21.15.17:0/1039975238 shutdown_connections 2024-03-16T10:11:35.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.768+0000 7f9891262640 1 -- 172.21.15.17:0/1039975238 wait complete. 2024-03-16T10:11:35.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.768+0000 7f9891262640 1 Processor -- start 2024-03-16T10:11:35.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.768+0000 7f9891262640 1 -- start start 2024-03-16T10:11:35.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f9891262640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f988c06d4f0 0x7f988c1489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f9891262640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f988c0b2080 0x7f988c1de0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f9891262640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f988c0b2530 0x7f988c1de5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f9891262640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f988c151c90 con 0x7f988c06d4f0 2024-03-16T10:11:35.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f9891262640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f988c151b10 con 0x7f988c0b2530 2024-03-16T10:11:35.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f9891262640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f988c151e10 con 0x7f988c0b2080 2024-03-16T10:11:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f988affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f988c06d4f0 0x7f988c1489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f988affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f988c06d4f0 0x7f988c1489b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51190/0 (socket says 172.21.15.17:51190) 2024-03-16T10:11:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f988affd640 1 -- 172.21.15.17:0/2588516352 learned_addr learned my addr 172.21.15.17:0/2588516352 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f988b7fe640 1 --2- 172.21.15.17:0/2588516352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f988c0b2530 0x7f988c1de5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f988affd640 1 -- 172.21.15.17:0/2588516352 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f988c0b2080 msgr2=0x7f988c1de0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f988affd640 1 --2- 172.21.15.17:0/2588516352 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f988c0b2080 0x7f988c1de0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f988affd640 1 -- 172.21.15.17:0/2588516352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f988c0b2530 msgr2=0x7f988c1de5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f988affd640 1 --2- 172.21.15.17:0/2588516352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f988c0b2530 0x7f988c1de5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f988affd640 1 -- 172.21.15.17:0/2588516352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f988c149cd0 con 0x7f988c06d4f0 2024-03-16T10:11:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f988b7fe640 1 --2- 172.21.15.17:0/2588516352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f988c0b2530 0x7f988c1de5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f988affd640 1 --2- 172.21.15.17:0/2588516352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f988c06d4f0 0x7f988c1489b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f987c00c420 tx=0x7f987c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:35.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f98837fe640 1 -- 172.21.15.17:0/2588516352 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c017070 con 0x7f988c06d4f0 2024-03-16T10:11:35.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f98837fe640 1 -- 172.21.15.17:0/2588516352 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f987c005cf0 con 0x7f988c06d4f0 2024-03-16T10:11:35.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f9891262640 1 -- 172.21.15.17:0/2588516352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f988c149fb0 con 0x7f988c06d4f0 2024-03-16T10:11:35.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f98837fe640 1 -- 172.21.15.17:0/2588516352 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f987c0072f0 con 0x7f988c06d4f0 2024-03-16T10:11:35.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f9891262640 1 -- 172.21.15.17:0/2588516352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f988c1e1e90 con 0x7f988c06d4f0 2024-03-16T10:11:35.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f9891262640 1 -- 172.21.15.17:0/2588516352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f988c1e19d0 con 0x7f988c06d4f0 2024-03-16T10:11:35.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f98837fe640 1 -- 172.21.15.17:0/2588516352 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f987c01a070 con 0x7f988c06d4f0 2024-03-16T10:11:35.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f98837fe640 1 --2- 172.21.15.17:0/2588516352 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f986c067690 0x7f986c069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:35.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f98837fe640 1 -- 172.21.15.17:0/2588516352 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7f987c084850 con 0x7f988c06d4f0 2024-03-16T10:11:35.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.772+0000 7f988a7fc640 1 --2- 172.21.15.17:0/2588516352 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f986c067690 0x7f986c069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:35.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.776+0000 7f988a7fc640 1 --2- 172.21.15.17:0/2588516352 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f986c067690 0x7f986c069b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f988c1470b0 tx=0x7f9874008330 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:35.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.776+0000 7f98837fe640 1 -- 172.21.15.17:0/2588516352 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f988c1e19d0 con 0x7f988c06d4f0 2024-03-16T10:11:35.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.992+0000 7f9891262640 1 -- 172.21.15.17:0/2588516352 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f988c068360 con 0x7f988c06d4f0 2024-03-16T10:11:36.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.996+0000 7f98837fe640 1 -- 172.21.15.17:0/2588516352 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f988c149cd0 con 0x7f988c06d4f0 2024-03-16T10:11:36.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.996+0000 7f9891262640 1 -- 172.21.15.17:0/2588516352 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f986c067690 msgr2=0x7f986c069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.996+0000 7f9891262640 1 --2- 172.21.15.17:0/2588516352 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f986c067690 0x7f986c069b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f988c1470b0 tx=0x7f9874008330 comp rx=0 tx=0).stop 2024-03-16T10:11:36.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.996+0000 7f9891262640 1 -- 172.21.15.17:0/2588516352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f988c06d4f0 msgr2=0x7f988c1489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:35.996+0000 7f9891262640 1 --2- 172.21.15.17:0/2588516352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f988c06d4f0 0x7f988c1489b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f987c00c420 tx=0x7f987c00c8e0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.000+0000 7f9891262640 1 -- 172.21.15.17:0/2588516352 shutdown_connections 2024-03-16T10:11:36.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.000+0000 7f9891262640 1 --2- 172.21.15.17:0/2588516352 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f986c067690 0x7f986c069b50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.000+0000 7f9891262640 1 --2- 172.21.15.17:0/2588516352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f988c0b2530 0x7f988c1de5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.000+0000 7f9891262640 1 --2- 172.21.15.17:0/2588516352 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f988c0b2080 0x7f988c1de0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.000+0000 7f9891262640 1 --2- 172.21.15.17:0/2588516352 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f988c06d4f0 0x7f988c1489b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.000+0000 7f9891262640 1 -- 172.21.15.17:0/2588516352 >> 172.21.15.17:0/2588516352 conn(0x7f988c0b55c0 msgr2=0x7f988c0b59a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.000+0000 7f9891262640 1 -- 172.21.15.17:0/2588516352 shutdown_connections 2024-03-16T10:11:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.000+0000 7f9891262640 1 -- 172.21.15.17:0/2588516352 wait complete. 2024-03-16T10:11:36.020 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:36.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.132+0000 7fb0cc8c2640 1 Processor -- start 2024-03-16T10:11:36.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.132+0000 7fb0cc8c2640 1 -- start start 2024-03-16T10:11:36.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0cc8c2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb0c806d750 0x7fb0c806f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0cc8c2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0c806db30 con 0x7fb0c80b2820 2024-03-16T10:11:36.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0cc8c2640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0c80b2d30 con 0x7fb0c8145280 2024-03-16T10:11:36.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0cc8c2640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb0c806fcf0 con 0x7fb0c806d750 2024-03-16T10:11:36.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0c5d9b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb0c80b2820 0x7fb0c813eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36202/0 (socket says 172.21.15.17:36202) 2024-03-16T10:11:36.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0c5d9b640 1 -- 172.21.15.17:0/1414611729 learned_addr learned my addr 172.21.15.17:0/1414611729 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:36.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0c659c640 1 --2- 172.21.15.17:0/1414611729 >> v2:172.21.15.17:3300/0 conn(0x7fb0c806d750 0x7fb0c806f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0c559a640 1 -- 172.21.15.17:0/1414611729 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2856251058 0 0) 0x7fb0c806db30 con 0x7fb0c80b2820 2024-03-16T10:11:36.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0c559a640 1 -- 172.21.15.17:0/1414611729 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0b0003530 con 0x7fb0c80b2820 2024-03-16T10:11:36.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0c659c640 1 -- 172.21.15.17:0/1414611729 >> v1:172.21.15.174:6789/0 conn(0x7fb0c8145280 legacy=0x7fb0c814fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:36.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0c659c640 1 -- 172.21.15.17:0/1414611729 >> v1:172.21.15.17:6789/0 conn(0x7fb0c80b2820 legacy=0x7fb0c813eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0c659c640 1 -- 172.21.15.17:0/1414611729 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c80afaf0 con 0x7fb0c806d750 2024-03-16T10:11:36.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0c659c640 1 --2- 172.21.15.17:0/1414611729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c806d750 0x7fb0c806f7b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80097c0 tx=0x7fb0b801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77431cb1d118efa server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:36.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0c559a640 1 -- 172.21.15.17:0/1414611729 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80b0070 con 0x7fb0c806d750 2024-03-16T10:11:36.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0c559a640 1 -- 172.21.15.17:0/1414611729 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b80ab600 con 0x7fb0c806d750 2024-03-16T10:11:36.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0c559a640 1 -- 172.21.15.17:0/1414611729 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80ab9d0 con 0x7fb0c806d750 2024-03-16T10:11:36.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/1414611729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c806d750 msgr2=0x7fb0c806f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0cc8c2640 1 --2- 172.21.15.17:0/1414611729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c806d750 0x7fb0c806f7b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80097c0 tx=0x7fb0b801c380 comp rx=0 tx=0).stop 2024-03-16T10:11:36.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/1414611729 shutdown_connections 2024-03-16T10:11:36.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0cc8c2640 1 --2- 172.21.15.17:0/1414611729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c806d750 0x7fb0c806f7b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/1414611729 >> 172.21.15.17:0/1414611729 conn(0x7fb0c80b55c0 msgr2=0x7fb0c80b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:36.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/1414611729 shutdown_connections 2024-03-16T10:11:36.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.136+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/1414611729 wait complete. 2024-03-16T10:11:36.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0cc8c2640 1 Processor -- start 2024-03-16T10:11:36.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0cc8c2640 1 -- start start 2024-03-16T10:11:36.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0cc8c2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c806d750 0x7fb0c81e3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0cc8c2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c80b2820 0x7fb0c8135ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0cc8c2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c8145280 0x7fb0c81dfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0cc8c2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c813fa00 con 0x7fb0c8145280 2024-03-16T10:11:36.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0cc8c2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c80b49f0 con 0x7fb0c806d750 2024-03-16T10:11:36.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0cc8c2640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c8153c70 con 0x7fb0c80b2820 2024-03-16T10:11:36.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0c6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c8145280 0x7fb0c81dfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0c6d9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c8145280 0x7fb0c81dfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51198/0 (socket says 172.21.15.17:51198) 2024-03-16T10:11:36.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0c6d9d640 1 -- 172.21.15.17:0/4034420905 learned_addr learned my addr 172.21.15.17:0/4034420905 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:36.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0c659c640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c806d750 0x7fb0c81e3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0c5d9b640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c80b2820 0x7fb0c8135ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0c6d9d640 1 -- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c80b2820 msgr2=0x7fb0c8135ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0c6d9d640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c80b2820 0x7fb0c8135ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0c6d9d640 1 -- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c806d750 msgr2=0x7fb0c81e3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0c6d9d640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c806d750 0x7fb0c81e3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0c6d9d640 1 -- 172.21.15.17:0/4034420905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c80b0b40 con 0x7fb0c8145280 2024-03-16T10:11:36.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0c659c640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c806d750 0x7fb0c81e3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:36.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0c5d9b640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c80b2820 0x7fb0c8135ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:36.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0c6d9d640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c8145280 0x7fb0c81dfd80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000c730 tx=0x7fb0c000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:36.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0b6ffd640 1 -- 172.21.15.17:0/4034420905 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0003b80 con 0x7fb0c8145280 2024-03-16T10:11:36.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0b6ffd640 1 -- 172.21.15.17:0/4034420905 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c0003d20 con 0x7fb0c8145280 2024-03-16T10:11:36.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/4034420905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c80af340 con 0x7fb0c8145280 2024-03-16T10:11:36.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0b6ffd640 1 -- 172.21.15.17:0/4034420905 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000aa30 con 0x7fb0c8145280 2024-03-16T10:11:36.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/4034420905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c81e3670 con 0x7fb0c8145280 2024-03-16T10:11:36.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/4034420905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c8068360 con 0x7fb0c8145280 2024-03-16T10:11:36.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.140+0000 7fb0b6ffd640 1 -- 172.21.15.17:0/4034420905 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fb0c000abd0 con 0x7fb0c8145280 2024-03-16T10:11:36.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.144+0000 7fb0b6ffd640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 0x7fb0a8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.144+0000 7fb0b6ffd640 1 -- 172.21.15.17:0/4034420905 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7fb0c0085360 con 0x7fb0c8145280 2024-03-16T10:11:36.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.144+0000 7fb0c659c640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 0x7fb0a8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.144+0000 7fb0c659c640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 0x7fb0a8069b20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80ad000 tx=0x7fb0b8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:36.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.148+0000 7fb0b6ffd640 1 -- 172.21.15.17:0/4034420905 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb0c00563c0 con 0x7fb0c8145280 2024-03-16T10:11:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.304+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/4034420905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb0c80b0860 con 0x7fb0c8145280 2024-03-16T10:11:36.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.304+0000 7fb0b6ffd640 1 -- 172.21.15.17:0/4034420905 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+12891 (secure 0 0 0) 0x7fb0c005c0c0 con 0x7fb0c8145280 2024-03-16T10:11:36.311 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:36.311 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":26,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:11:33.331607+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:36.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.308+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 msgr2=0x7fb0a8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.308+0000 7fb0cc8c2640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 0x7fb0a8069b20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80ad000 tx=0x7fb0b8008040 comp rx=0 tx=0).stop 2024-03-16T10:11:36.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.308+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/4034420905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c8145280 msgr2=0x7fb0c81dfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.308+0000 7fb0cc8c2640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c8145280 0x7fb0c81dfd80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000c730 tx=0x7fb0c000cbf0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.308+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/4034420905 shutdown_connections 2024-03-16T10:11:36.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.308+0000 7fb0cc8c2640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 0x7fb0a8069b20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.308+0000 7fb0cc8c2640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c8145280 0x7fb0c81dfd80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.308+0000 7fb0cc8c2640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c80b2820 0x7fb0c8135ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.308+0000 7fb0cc8c2640 1 --2- 172.21.15.17:0/4034420905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c806d750 0x7fb0c81e3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.308+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/4034420905 >> 172.21.15.17:0/4034420905 conn(0x7fb0c80b55c0 msgr2=0x7fb0c80b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.308+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/4034420905 shutdown_connections 2024-03-16T10:11:36.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.308+0000 7fb0cc8c2640 1 -- 172.21.15.17:0/4034420905 wait complete. 2024-03-16T10:11:36.328 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:11:36.329 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:11:36.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f38a14e3640 1 Processor -- start 2024-03-16T10:11:36.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f38a14e3640 1 -- start start 2024-03-16T10:11:36.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f38a14e3640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f389c0b4200 0x7f389c150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f38a14e3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f389c0af0c0 con 0x7f389c0ae060 2024-03-16T10:11:36.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f38a14e3640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f389c0b27e0 con 0x7f389c0b2400 2024-03-16T10:11:36.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f38a14e3640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f389c0afcf0 con 0x7f389c0b4200 2024-03-16T10:11:36.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f3893fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f389c0ae060 0x7f389c072b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36208/0 (socket says 172.21.15.17:36208) 2024-03-16T10:11:36.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f3893fff640 1 -- 172.21.15.17:0/4103183988 learned_addr learned my addr 172.21.15.17:0/4103183988 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:36.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389b7fe640 1 --2- 172.21.15.17:0/4103183988 >> v2:172.21.15.174:3300/0 conn(0x7f389c0b4200 0x7f389c150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 -- 172.21.15.17:0/4103183988 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3432766825 0 0) 0x7f389c0af0c0 con 0x7f389c0ae060 2024-03-16T10:11:36.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 -- 172.21.15.17:0/4103183988 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3888003530 con 0x7f389c0ae060 2024-03-16T10:11:36.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 -- 172.21.15.17:0/4103183988 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3456748176 0 0) 0x7f3888003530 con 0x7f389c0ae060 2024-03-16T10:11:36.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 -- 172.21.15.17:0/4103183988 >> v2:172.21.15.174:3300/0 conn(0x7f389c0b4200 msgr2=0x7f389c150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:36.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 --2- 172.21.15.17:0/4103183988 >> v2:172.21.15.174:3300/0 conn(0x7f389c0b4200 0x7f389c150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 -- 172.21.15.17:0/4103183988 >> v1:172.21.15.174:6789/0 conn(0x7f389c0b2400 legacy=0x7f389c1452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 -- 172.21.15.17:0/4103183988 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c0af980 con 0x7f389c0ae060 2024-03-16T10:11:36.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 -- 172.21.15.17:0/4103183988 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f3884001680 con 0x7f389c0ae060 2024-03-16T10:11:36.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 -- 172.21.15.17:0/4103183988 >> v1:172.21.15.17:6789/0 conn(0x7f389c0ae060 legacy=0x7f389c072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 --2- 172.21.15.17:0/4103183988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880039c0 0x7f3888003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 --2- 172.21.15.17:0/4103183988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3888004810 0x7f3888004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 --2- 172.21.15.17:0/4103183988 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38880051d0 0x7f388800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 -- 172.21.15.17:0/4103183988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f389c0afcf0 con 0x7f38880039c0 2024-03-16T10:11:36.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 -- 172.21.15.17:0/4103183988 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f388800c070 con 0x7f3888004810 2024-03-16T10:11:36.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389a7fc640 1 -- 172.21.15.17:0/4103183988 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f388800c270 con 0x7f38880051d0 2024-03-16T10:11:36.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389affd640 1 --2- 172.21.15.17:0/4103183988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880039c0 0x7f3888003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389affd640 1 -- 172.21.15.17:0/4103183988 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38880051d0 msgr2=0x7f388800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:36.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389affd640 1 --2- 172.21.15.17:0/4103183988 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f38880051d0 0x7f388800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389affd640 1 -- 172.21.15.17:0/4103183988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3888004810 msgr2=0x7f3888004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:36.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389affd640 1 --2- 172.21.15.17:0/4103183988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3888004810 0x7f3888004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389affd640 1 -- 172.21.15.17:0/4103183988 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3888003530 con 0x7f38880039c0 2024-03-16T10:11:36.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.440+0000 7f389b7fe640 1 -- 172.21.15.17:0/4103183988 reap_dead start 2024-03-16T10:11:36.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f389affd640 1 --2- 172.21.15.17:0/4103183988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880039c0 0x7f3888003de0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f388c001780 tx=0x7f388c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e01b40debb858d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:36.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f389a7fc640 1 -- 172.21.15.17:0/4103183988 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c0013a0 con 0x7f38880039c0 2024-03-16T10:11:36.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f389a7fc640 1 -- 172.21.15.17:0/4103183988 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388c0052f0 con 0x7f38880039c0 2024-03-16T10:11:36.452 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f389a7fc640 1 -- 172.21.15.17:0/4103183988 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c003840 con 0x7f38880039c0 2024-03-16T10:11:36.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 -- 172.21.15.17:0/4103183988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880039c0 msgr2=0x7f3888003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 --2- 172.21.15.17:0/4103183988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880039c0 0x7f3888003de0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f388c001780 tx=0x7f388c01c3d0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 -- 172.21.15.17:0/4103183988 shutdown_connections 2024-03-16T10:11:36.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 --2- 172.21.15.17:0/4103183988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880039c0 0x7f3888003de0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 -- 172.21.15.17:0/4103183988 >> 172.21.15.17:0/4103183988 conn(0x7f389c0b8e90 msgr2=0x7f389c0b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:36.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 -- 172.21.15.17:0/4103183988 shutdown_connections 2024-03-16T10:11:36.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 -- 172.21.15.17:0/4103183988 wait complete. 2024-03-16T10:11:36.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 Processor -- start 2024-03-16T10:11:36.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 -- start start 2024-03-16T10:11:36.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880039c0 0x7f389c142760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f389c1ddec0 0x7f389c148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f389c13fc30 0x7f389c071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f389c072f10 con 0x7f38880039c0 2024-03-16T10:11:36.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f389c072d90 con 0x7f389c1ddec0 2024-03-16T10:11:36.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f389c073090 con 0x7f389c13fc30 2024-03-16T10:11:36.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f389affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880039c0 0x7f389c142760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f389affd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880039c0 0x7f389c142760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51218/0 (socket says 172.21.15.17:51218) 2024-03-16T10:11:36.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f389affd640 1 -- 172.21.15.17:0/765216083 learned_addr learned my addr 172.21.15.17:0/765216083 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:36.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f389b7fe640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f389c13fc30 0x7f389c071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f3893fff640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f389c1ddec0 0x7f389c148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f389affd640 1 -- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f389c13fc30 msgr2=0x7f389c071ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f389affd640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f389c13fc30 0x7f389c071ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f389affd640 1 -- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f389c1ddec0 msgr2=0x7f389c148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f389affd640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f389c1ddec0 0x7f389c148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f389affd640 1 -- 172.21.15.17:0/765216083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c148eb0 con 0x7f38880039c0 2024-03-16T10:11:36.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f3893fff640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f389c1ddec0 0x7f389c148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:36.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f389affd640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880039c0 0x7f389c142760 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f388c01c200 tx=0x7f3888005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:36.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38937fe640 1 -- 172.21.15.17:0/765216083 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c0ba070 con 0x7f38880039c0 2024-03-16T10:11:36.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38937fe640 1 -- 172.21.15.17:0/765216083 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388c0054a0 con 0x7f38880039c0 2024-03-16T10:11:36.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 -- 172.21.15.17:0/765216083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389c1de3c0 con 0x7f38880039c0 2024-03-16T10:11:36.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38937fe640 1 -- 172.21.15.17:0/765216083 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c0013a0 con 0x7f38880039c0 2024-03-16T10:11:36.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.444+0000 7f38a14e3640 1 -- 172.21.15.17:0/765216083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f389c1400e0 con 0x7f38880039c0 2024-03-16T10:11:36.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.448+0000 7f38a14e3640 1 -- 172.21.15.17:0/765216083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389c068360 con 0x7f38880039c0 2024-03-16T10:11:36.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.448+0000 7f38937fe640 1 -- 172.21.15.17:0/765216083 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f388c0bd070 con 0x7f38880039c0 2024-03-16T10:11:36.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.448+0000 7f38937fe640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f387c067690 0x7f387c069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.448+0000 7f38937fe640 1 -- 172.21.15.17:0/765216083 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7f388c11e160 con 0x7f38880039c0 2024-03-16T10:11:36.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.448+0000 7f3893fff640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f387c067690 0x7f387c069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.448+0000 7f3893fff640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f387c067690 0x7f387c069b50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f38840011f0 tx=0x7f3884009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:36.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.452+0000 7f38937fe640 1 -- 172.21.15.17:0/765216083 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f388c016110 con 0x7f38880039c0 2024-03-16T10:11:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.612+0000 7f38a14e3640 1 -- 172.21.15.17:0/765216083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f389c0b3fe0 con 0x7f38880039c0 2024-03-16T10:11:36.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.612+0000 7f38937fe640 1 -- 172.21.15.17:0/765216083 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f388c0ef280 con 0x7f38880039c0 2024-03-16T10:11:36.619 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:11:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.616+0000 7f38a14e3640 1 -- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f387c067690 msgr2=0x7f387c069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.616+0000 7f38a14e3640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f387c067690 0x7f387c069b50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f38840011f0 tx=0x7f3884009040 comp rx=0 tx=0).stop 2024-03-16T10:11:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.616+0000 7f38a14e3640 1 -- 172.21.15.17:0/765216083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880039c0 msgr2=0x7f389c142760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.616+0000 7f38a14e3640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880039c0 0x7f389c142760 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f388c01c200 tx=0x7f3888005840 comp rx=0 tx=0).stop 2024-03-16T10:11:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.616+0000 7f38a14e3640 1 -- 172.21.15.17:0/765216083 shutdown_connections 2024-03-16T10:11:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.616+0000 7f38a14e3640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f387c067690 0x7f387c069b50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.616+0000 7f38a14e3640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f389c13fc30 0x7f389c071ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.616+0000 7f38a14e3640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f389c1ddec0 0x7f389c148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.616+0000 7f38a14e3640 1 --2- 172.21.15.17:0/765216083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f38880039c0 0x7f389c142760 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.616+0000 7f38a14e3640 1 -- 172.21.15.17:0/765216083 >> 172.21.15.17:0/765216083 conn(0x7f389c0b8e90 msgr2=0x7f389c13eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.616+0000 7f38a14e3640 1 -- 172.21.15.17:0/765216083 shutdown_connections 2024-03-16T10:11:36.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.616+0000 7f38a14e3640 1 -- 172.21.15.17:0/765216083 wait complete. 2024-03-16T10:11:36.638 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-16T10:11:36.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.748+0000 7f6ab319b640 1 Processor -- start 2024-03-16T10:11:36.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.748+0000 7f6ab319b640 1 -- start start 2024-03-16T10:11:36.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.752+0000 7f6ab319b640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f6aac06d750 0x7f6aac06f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.752+0000 7f6ab319b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6aac145280 0x7f6aac13f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.752+0000 7f6ab319b640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aac06db30 con 0x7f6aac0b2820 2024-03-16T10:11:36.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.752+0000 7f6ab319b640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f6aac0b2d30 con 0x7f6aac06d750 2024-03-16T10:11:36.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.752+0000 7f6ab319b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f6aac06fcf0 con 0x7f6aac145280 2024-03-16T10:11:36.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.752+0000 7f6ab1738640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6aac145280 0x7f6aac13f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.752+0000 7f6ab1738640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6aac145280 0x7f6aac13f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.17:33704/0 (socket says 172.21.15.17:33704) 2024-03-16T10:11:36.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.752+0000 7f6ab1738640 1 -- 172.21.15.17:0/274395256 learned_addr learned my addr 172.21.15.17:0/274395256 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:36.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.752+0000 7f6ab0f37640 1 --2- 172.21.15.17:0/274395256 >> v2:172.21.15.174:3300/0 conn(0x7f6aac06d750 0x7f6aac06f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.752+0000 7f6ab1738640 1 -- 172.21.15.17:0/274395256 >> v2:172.21.15.174:3300/0 conn(0x7f6aac06d750 msgr2=0x7f6aac06f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:36.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.752+0000 7f6ab1738640 1 --2- 172.21.15.17:0/274395256 >> v2:172.21.15.174:3300/0 conn(0x7f6aac06d750 0x7f6aac06f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.752+0000 7f6ab1738640 1 -- 172.21.15.17:0/274395256 >> v1:172.21.15.174:6789/0 conn(0x7f6aac0b2820 legacy=0x7f6aac13eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.752+0000 7f6ab1738640 1 -- 172.21.15.17:0/274395256 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac0afaf0 con 0x7f6aac145280 2024-03-16T10:11:36.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.752+0000 7f6ab1738640 1 --2- 172.21.15.17:0/274395256 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6aac145280 0x7f6aac13f5f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009e50 tx=0x7f6aa401c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89bae46472e7cb5f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:36.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6aab7fe640 1 -- 172.21.15.17:0/274395256 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40a55d0 con 0x7f6aac145280 2024-03-16T10:11:36.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6aab7fe640 1 -- 172.21.15.17:0/274395256 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa40a5770 con 0x7f6aac145280 2024-03-16T10:11:36.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6aab7fe640 1 -- 172.21.15.17:0/274395256 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40a5b40 con 0x7f6aac145280 2024-03-16T10:11:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 -- 172.21.15.17:0/274395256 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6aac145280 msgr2=0x7f6aac13f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 --2- 172.21.15.17:0/274395256 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6aac145280 0x7f6aac13f5f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009e50 tx=0x7f6aa401c910 comp rx=0 tx=0).stop 2024-03-16T10:11:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 -- 172.21.15.17:0/274395256 shutdown_connections 2024-03-16T10:11:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 --2- 172.21.15.17:0/274395256 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6aac145280 0x7f6aac13f5f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 --2- 172.21.15.17:0/274395256 >> v2:172.21.15.174:3300/0 conn(0x7f6aac06d750 0x7f6aac06f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 -- 172.21.15.17:0/274395256 >> 172.21.15.17:0/274395256 conn(0x7f6aac0b55c0 msgr2=0x7f6aac0b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 -- 172.21.15.17:0/274395256 shutdown_connections 2024-03-16T10:11:36.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 -- 172.21.15.17:0/274395256 wait complete. 2024-03-16T10:11:36.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 Processor -- start 2024-03-16T10:11:36.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 -- start start 2024-03-16T10:11:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6aac06d750 0x7f6aac138fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6aac0b2820 0x7f6aac1df960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aac145280 0x7f6aac1dfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac152550 con 0x7f6aac145280 2024-03-16T10:11:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6aac0b49f0 con 0x7f6aac06d750 2024-03-16T10:11:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab319b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f6aac1526d0 con 0x7f6aac0b2820 2024-03-16T10:11:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab1738640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aac145280 0x7f6aac1dfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab1738640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aac145280 0x7f6aac1dfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51230/0 (socket says 172.21.15.17:51230) 2024-03-16T10:11:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab1738640 1 -- 172.21.15.17:0/2234761722 learned_addr learned my addr 172.21.15.17:0/2234761722 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6ab0f37640 1 --2- 172.21.15.17:0/2234761722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6aac06d750 0x7f6aac138fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.756+0000 7f6aabfff640 1 --2- 172.21.15.17:0/2234761722 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6aac0b2820 0x7f6aac1df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6ab1738640 1 -- 172.21.15.17:0/2234761722 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6aac0b2820 msgr2=0x7f6aac1df960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6ab1738640 1 --2- 172.21.15.17:0/2234761722 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6aac0b2820 0x7f6aac1df960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6ab1738640 1 -- 172.21.15.17:0/2234761722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6aac06d750 msgr2=0x7f6aac138fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6ab1738640 1 --2- 172.21.15.17:0/2234761722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6aac06d750 0x7f6aac138fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6ab1738640 1 -- 172.21.15.17:0/2234761722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aac0b1010 con 0x7f6aac145280 2024-03-16T10:11:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6ab1738640 1 --2- 172.21.15.17:0/2234761722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aac145280 0x7f6aac1dfea0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40af040 tx=0x7f6aa40196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6aa97fa640 1 -- 172.21.15.17:0/2234761722 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40bc020 con 0x7f6aac145280 2024-03-16T10:11:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6aa97fa640 1 -- 172.21.15.17:0/2234761722 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa40b88c0 con 0x7f6aac145280 2024-03-16T10:11:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6ab319b640 1 -- 172.21.15.17:0/2234761722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aac0b12f0 con 0x7f6aac145280 2024-03-16T10:11:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6aa97fa640 1 -- 172.21.15.17:0/2234761722 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40a55d0 con 0x7f6aac145280 2024-03-16T10:11:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6ab319b640 1 -- 172.21.15.17:0/2234761722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aac139aa0 con 0x7f6aac145280 2024-03-16T10:11:36.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6ab319b640 1 -- 172.21.15.17:0/2234761722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aac068360 con 0x7f6aac145280 2024-03-16T10:11:36.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6aa97fa640 1 -- 172.21.15.17:0/2234761722 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f6aa40bf020 con 0x7f6aac145280 2024-03-16T10:11:36.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6aa97fa640 1 --2- 172.21.15.17:0/2234761722 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6a98067690 0x7f6a98069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:36.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6aa97fa640 1 -- 172.21.15.17:0/2234761722 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7f6aa411ff90 con 0x7f6aac145280 2024-03-16T10:11:36.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6ab0f37640 1 --2- 172.21.15.17:0/2234761722 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6a98067690 0x7f6a98069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:36.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.760+0000 7f6ab0f37640 1 --2- 172.21.15.17:0/2234761722 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6a98067690 0x7f6a98069b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009b60 tx=0x7f6a9c009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:36.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.764+0000 7f6aa97fa640 1 -- 172.21.15.17:0/2234761722 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6aa40c6020 con 0x7f6aac145280 2024-03-16T10:11:36.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.928+0000 7f6ab319b640 1 -- 172.21.15.17:0/2234761722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6aac142ed0 con 0x7f6aac145280 2024-03-16T10:11:36.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.928+0000 7f6aa97fa640 1 -- 172.21.15.17:0/2234761722 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v26) ==== 84+0+65 (secure 0 0 0) 0x7f6aa4016620 con 0x7f6aac145280 2024-03-16T10:11:36.935 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:36.935 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-03-16T10:11:36.935 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-03-16T10:11:36.935 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-03-16T10:11:36.935 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-03-16T10:11:36.935 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:36.935 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-03-16T10:11:36.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.932+0000 7f6ab319b640 1 -- 172.21.15.17:0/2234761722 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6a98067690 msgr2=0x7f6a98069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.932+0000 7f6ab319b640 1 --2- 172.21.15.17:0/2234761722 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6a98067690 0x7f6a98069b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c009b60 tx=0x7f6a9c009190 comp rx=0 tx=0).stop 2024-03-16T10:11:36.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.932+0000 7f6ab319b640 1 -- 172.21.15.17:0/2234761722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aac145280 msgr2=0x7f6aac1dfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:36.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.932+0000 7f6ab319b640 1 --2- 172.21.15.17:0/2234761722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aac145280 0x7f6aac1dfea0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40af040 tx=0x7f6aa40196b0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.932+0000 7f6ab319b640 1 -- 172.21.15.17:0/2234761722 shutdown_connections 2024-03-16T10:11:36.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.932+0000 7f6ab319b640 1 --2- 172.21.15.17:0/2234761722 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f6a98067690 0x7f6a98069b50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.932+0000 7f6ab319b640 1 --2- 172.21.15.17:0/2234761722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6aac145280 0x7f6aac1dfea0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.932+0000 7f6ab319b640 1 --2- 172.21.15.17:0/2234761722 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f6aac0b2820 0x7f6aac1df960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.932+0000 7f6ab319b640 1 --2- 172.21.15.17:0/2234761722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6aac06d750 0x7f6aac138fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:36.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.932+0000 7f6ab319b640 1 -- 172.21.15.17:0/2234761722 >> 172.21.15.17:0/2234761722 conn(0x7f6aac0b55c0 msgr2=0x7f6aac13a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:36.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.932+0000 7f6ab319b640 1 -- 172.21.15.17:0/2234761722 shutdown_connections 2024-03-16T10:11:36.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:36.932+0000 7f6ab319b640 1 -- 172.21.15.17:0/2234761722 wait complete. 2024-03-16T10:11:36.954 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-16T10:11:36.954 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-16T10:11:37.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.072+0000 7fcd805c5640 1 Processor -- start 2024-03-16T10:11:37.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.072+0000 7fcd805c5640 1 -- start start 2024-03-16T10:11:37.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.072+0000 7fcd805c5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd780b6c20 0x7fcd78145270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:37.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.072+0000 7fcd805c5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd780b59c0 con 0x7fcd780b5c00 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.072+0000 7fcd805c5640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd780b7ac0 con 0x7fcd780b49d0 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.072+0000 7fcd805c5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fcd780b7da0 con 0x7fcd780b6c20 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.072+0000 7fcd7db60640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fcd780b5c00 0x7fcd780b42c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:36216/0 (socket says 172.21.15.17:36216) 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.072+0000 7fcd7db60640 1 -- 172.21.15.17:0/441842612 learned_addr learned my addr 172.21.15.17:0/441842612 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.072+0000 7fcd7e361640 1 --2- 172.21.15.17:0/441842612 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd780b6c20 0x7fcd78145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.072+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 197534286 0 0) 0x7fcd780b59c0 con 0x7fcd780b5c00 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.072+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd64003530 con 0x7fcd780b5c00 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.072+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 <== mon.1 v1:172.21.15.174:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2203686171 0 0) 0x7fcd780b7ac0 con 0x7fcd780b49d0 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.072+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 --> v1:172.21.15.174:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcd780b59c0 con 0x7fcd780b49d0 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.072+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 408075441 0 0) 0x7fcd64003530 con 0x7fcd780b5c00 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd780b6c20 msgr2=0x7fcd78145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 --2- 172.21.15.17:0/441842612 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd780b6c20 0x7fcd78145270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 >> v1:172.21.15.174:6789/0 conn(0x7fcd780b49d0 legacy=0x7fcd7814fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd780b7000 con 0x7fcd780b5c00 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fcd68001680 con 0x7fcd780b5c00 2024-03-16T10:11:37.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 >> v1:172.21.15.17:6789/0 conn(0x7fcd780b5c00 legacy=0x7fcd780b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 --2- 172.21.15.17:0/441842612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd64003b80 0x7fcd64003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 --2- 172.21.15.17:0/441842612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd64004810 0x7fcd64004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 --2- 172.21.15.17:0/441842612 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd780b6c20 0x7fcd64005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcd780b7da0 con 0x7fcd64004810 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd6400bb90 con 0x7fcd64003b80 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fcd6400bd90 con 0x7fcd780b6c20 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7db60640 1 --2- 172.21.15.17:0/441842612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd64004810 0x7fcd64004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7e361640 1 --2- 172.21.15.17:0/441842612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd64003b80 0x7fcd64003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7eb62640 1 --2- 172.21.15.17:0/441842612 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd780b6c20 0x7fcd64005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7db60640 1 -- 172.21.15.17:0/441842612 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd780b6c20 msgr2=0x7fcd64005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7db60640 1 --2- 172.21.15.17:0/441842612 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd780b6c20 0x7fcd64005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7db60640 1 -- 172.21.15.17:0/441842612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd64003b80 msgr2=0x7fcd64003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7db60640 1 --2- 172.21.15.17:0/441842612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd64003b80 0x7fcd64003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7db60640 1 -- 172.21.15.17:0/441842612 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd64003530 con 0x7fcd64004810 2024-03-16T10:11:37.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7eb62640 1 --2- 172.21.15.17:0/441842612 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd780b6c20 0x7fcd64005280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:37.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7db60640 1 --2- 172.21.15.17:0/441842612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd64004810 0x7fcd64004c90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800c1c0 tx=0x7fcd6801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd90e9fdf3a36d0f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:37.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68007cf0 con 0x7fcd64004810 2024-03-16T10:11:37.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd680a9020 con 0x7fcd64004810 2024-03-16T10:11:37.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7d35f640 1 -- 172.21.15.17:0/441842612 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd68005b00 con 0x7fcd64004810 2024-03-16T10:11:37.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd805c5640 1 -- 172.21.15.17:0/441842612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd64004810 msgr2=0x7fcd64004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:37.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd805c5640 1 --2- 172.21.15.17:0/441842612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd64004810 0x7fcd64004c90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcd6800c1c0 tx=0x7fcd6801cc20 comp rx=0 tx=0).stop 2024-03-16T10:11:37.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd7eb62640 1 -- 172.21.15.17:0/441842612 reap_dead start 2024-03-16T10:11:37.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd805c5640 1 -- 172.21.15.17:0/441842612 shutdown_connections 2024-03-16T10:11:37.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd805c5640 1 -- 172.21.15.17:0/441842612 >> 172.21.15.17:0/441842612 conn(0x7fcd7806b460 msgr2=0x7fcd7806b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:37.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd805c5640 1 -- 172.21.15.17:0/441842612 shutdown_connections 2024-03-16T10:11:37.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.076+0000 7fcd805c5640 1 -- 172.21.15.17:0/441842612 wait complete. 2024-03-16T10:11:37.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd805c5640 1 Processor -- start 2024-03-16T10:11:37.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd805c5640 1 -- start start 2024-03-16T10:11:37.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd805c5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd64004810 0x7fcd78150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd805c5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd7806db20 0x7fcd781dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd805c5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd781e31a0 0x7fcd781e3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd805c5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcd781515c0 con 0x7fcd64004810 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd805c5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78151440 con 0x7fcd781e31a0 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd805c5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fcd78151740 con 0x7fcd7806db20 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd7e361640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd64004810 0x7fcd78150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd7e361640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd64004810 0x7fcd78150160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51242/0 (socket says 172.21.15.17:51242) 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd7e361640 1 -- 172.21.15.17:0/2956839884 learned_addr learned my addr 172.21.15.17:0/2956839884 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd7db60640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd7806db20 0x7fcd781dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd7eb62640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd781e31a0 0x7fcd781e3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd7e361640 1 -- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd7806db20 msgr2=0x7fcd781dfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd7e361640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd7806db20 0x7fcd781dfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd7e361640 1 -- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd781e31a0 msgr2=0x7fcd781e3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd7e361640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd781e31a0 0x7fcd781e3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd7e361640 1 -- 172.21.15.17:0/2956839884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd780bbaa0 con 0x7fcd64004810 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd7db60640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd7806db20 0x7fcd781dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:37.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd7e361640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd64004810 0x7fcd78150160 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcd740115e0 tx=0x7fcd74011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:37.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd6effd640 1 -- 172.21.15.17:0/2956839884 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7400f8f0 con 0x7fcd64004810 2024-03-16T10:11:37.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd6effd640 1 -- 172.21.15.17:0/2956839884 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd7400fa90 con 0x7fcd64004810 2024-03-16T10:11:37.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd805c5640 1 -- 172.21.15.17:0/2956839884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7806d360 con 0x7fcd64004810 2024-03-16T10:11:37.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd6effd640 1 -- 172.21.15.17:0/2956839884 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7400fe40 con 0x7fcd64004810 2024-03-16T10:11:37.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd805c5640 1 -- 172.21.15.17:0/2956839884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7806ccb0 con 0x7fcd64004810 2024-03-16T10:11:37.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd805c5640 1 -- 172.21.15.17:0/2956839884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7806a130 con 0x7fcd64004810 2024-03-16T10:11:37.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd6effd640 1 -- 172.21.15.17:0/2956839884 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fcd74015460 con 0x7fcd64004810 2024-03-16T10:11:37.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.080+0000 7fcd6effd640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fcd5c067690 0x7fcd5c069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:37.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.084+0000 7fcd6effd640 1 -- 172.21.15.17:0/2956839884 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7fcd74085da0 con 0x7fcd64004810 2024-03-16T10:11:37.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.084+0000 7fcd7db60640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fcd5c067690 0x7fcd5c069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:37.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.084+0000 7fcd7db60640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fcd5c067690 0x7fcd5c069b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcd680aa000 tx=0x7fcd68001a40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:37.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.088+0000 7fcd6effd640 1 -- 172.21.15.17:0/2956839884 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcd74019030 con 0x7fcd64004810 2024-03-16T10:11:37.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:37.240+0000 7fcd805c5640 1 -- 172.21.15.17:0/2956839884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcd78140140 con 0x7fcd64004810 2024-03-16T10:11:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.000+0000 7fcd6effd640 1 -- 172.21.15.17:0/2956839884 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v27) ==== 138+0+0 (secure 0 0 0) 0x7fcd74056e00 con 0x7fcd64004810 2024-03-16T10:11:38.007 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-03-16T10:11:38.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.004+0000 7fcd805c5640 1 -- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fcd5c067690 msgr2=0x7fcd5c069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.004+0000 7fcd805c5640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fcd5c067690 0x7fcd5c069b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcd680aa000 tx=0x7fcd68001a40 comp rx=0 tx=0).stop 2024-03-16T10:11:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.004+0000 7fcd805c5640 1 -- 172.21.15.17:0/2956839884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd64004810 msgr2=0x7fcd78150160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.004+0000 7fcd805c5640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd64004810 0x7fcd78150160 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcd740115e0 tx=0x7fcd74011aa0 comp rx=0 tx=0).stop 2024-03-16T10:11:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.004+0000 7fcd805c5640 1 -- 172.21.15.17:0/2956839884 shutdown_connections 2024-03-16T10:11:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.004+0000 7fcd805c5640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fcd5c067690 0x7fcd5c069b50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.004+0000 7fcd805c5640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd781e31a0 0x7fcd781e3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.004+0000 7fcd805c5640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fcd7806db20 0x7fcd781dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.004+0000 7fcd805c5640 1 --2- 172.21.15.17:0/2956839884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd64004810 0x7fcd78150160 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.004+0000 7fcd805c5640 1 -- 172.21.15.17:0/2956839884 >> 172.21.15.17:0/2956839884 conn(0x7fcd7806b460 msgr2=0x7fcd78072310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.004+0000 7fcd805c5640 1 -- 172.21.15.17:0/2956839884 shutdown_connections 2024-03-16T10:11:38.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.004+0000 7fcd805c5640 1 -- 172.21.15.17:0/2956839884 wait complete. 2024-03-16T10:11:38.035 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-16T10:11:38.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c32271640 1 Processor -- start 2024-03-16T10:11:38.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c32271640 1 -- start start 2024-03-16T10:11:38.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c32271640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0c2c0b5920 0x7f0c2c144410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:38.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c32271640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f0c2c0b40f0 0x7f0c2c144950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:38.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c32271640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c2c067600 con 0x7f0c2c067220 2024-03-16T10:11:38.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c32271640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f0c2c0b5110 con 0x7f0c2c0b5920 2024-03-16T10:11:38.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c32271640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f0c2c0ae8b0 con 0x7f0c2c0b40f0 2024-03-16T10:11:38.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c2b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0c2c0b5920 0x7f0c2c144410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:38.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c2b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0c2c0b5920 0x7f0c2c144410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51248/0 (socket says 172.21.15.17:51248) 2024-03-16T10:11:38.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c2b7fe640 1 -- 172.21.15.17:0/2128978979 learned_addr learned my addr 172.21.15.17:0/2128978979 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:38.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c3080e640 1 --2- 172.21.15.17:0/2128978979 >> v2:172.21.15.174:3300/0 conn(0x7f0c2c0b40f0 0x7f0c2c144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:38.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c2b7fe640 1 -- 172.21.15.17:0/2128978979 >> v2:172.21.15.174:3300/0 conn(0x7f0c2c0b40f0 msgr2=0x7f0c2c144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:38.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c2b7fe640 1 --2- 172.21.15.17:0/2128978979 >> v2:172.21.15.174:3300/0 conn(0x7f0c2c0b40f0 0x7f0c2c144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:38.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c2b7fe640 1 -- 172.21.15.17:0/2128978979 >> v1:172.21.15.174:6789/0 conn(0x7f0c2c067220 legacy=0x7f0c2c147540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:38.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.168+0000 7f0c2b7fe640 1 -- 172.21.15.17:0/2128978979 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2c0aeb90 con 0x7f0c2c0b5920 2024-03-16T10:11:38.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c2b7fe640 1 --2- 172.21.15.17:0/2128978979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c2c0b5920 0x7f0c2c144410 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c073880 tx=0x7f0c2001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab0424328dd781b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:38.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c2affd640 1 -- 172.21.15.17:0/2128978979 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c200b2070 con 0x7f0c2c0b5920 2024-03-16T10:11:38.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c2affd640 1 -- 172.21.15.17:0/2128978979 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c200ad640 con 0x7f0c2c0b5920 2024-03-16T10:11:38.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 -- 172.21.15.17:0/2128978979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c2c0b5920 msgr2=0x7f0c2c144410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:38.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 --2- 172.21.15.17:0/2128978979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c2c0b5920 0x7f0c2c144410 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c073880 tx=0x7f0c2001c390 comp rx=0 tx=0).stop 2024-03-16T10:11:38.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 -- 172.21.15.17:0/2128978979 shutdown_connections 2024-03-16T10:11:38.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 --2- 172.21.15.17:0/2128978979 >> v2:172.21.15.174:3300/0 conn(0x7f0c2c0b40f0 0x7f0c2c144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:38.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 --2- 172.21.15.17:0/2128978979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c2c0b5920 0x7f0c2c144410 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:38.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 -- 172.21.15.17:0/2128978979 >> 172.21.15.17:0/2128978979 conn(0x7f0c2c06b290 msgr2=0x7f0c2c06b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:38.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 -- 172.21.15.17:0/2128978979 shutdown_connections 2024-03-16T10:11:38.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 -- 172.21.15.17:0/2128978979 wait complete. 2024-03-16T10:11:38.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 Processor -- start 2024-03-16T10:11:38.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 -- start start 2024-03-16T10:11:38.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c2c067220 0x7f0c2c0af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:38.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c2c0b40f0 0x7f0c2c0afa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:38.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c2c0b5920 0x7f0c2c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:38.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c154a90 con 0x7f0c2c0b40f0 2024-03-16T10:11:38.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c2c1452a0 con 0x7f0c2c067220 2024-03-16T10:11:38.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.172+0000 7f0c32271640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f0c2c154c10 con 0x7f0c2c0b5920 2024-03-16T10:11:38.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c2b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c2c0b40f0 0x7f0c2c0afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:38.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c2b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c2c0b40f0 0x7f0c2c0afa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51252/0 (socket says 172.21.15.17:51252) 2024-03-16T10:11:38.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c2b7fe640 1 -- 172.21.15.17:0/181485187 learned_addr learned my addr 172.21.15.17:0/181485187 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:38.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c2bfff640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c2c067220 0x7f0c2c0af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:38.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c3080e640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c2c0b5920 0x7f0c2c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:38.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c2b7fe640 1 -- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c2c0b5920 msgr2=0x7f0c2c0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:38.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c2b7fe640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c2c0b5920 0x7f0c2c0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:38.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c2b7fe640 1 -- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c2c067220 msgr2=0x7f0c2c0af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:38.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c2b7fe640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c2c067220 0x7f0c2c0af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:38.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c2b7fe640 1 -- 172.21.15.17:0/181485187 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2c0b2af0 con 0x7f0c2c0b40f0 2024-03-16T10:11:38.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c2bfff640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c2c067220 0x7f0c2c0af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:38.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c2b7fe640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c2c0b40f0 0x7f0c2c0afa50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0c200b0040 tx=0x7f0c20019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:38.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c28ff9640 1 -- 172.21.15.17:0/181485187 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c200bd070 con 0x7f0c2c0b40f0 2024-03-16T10:11:38.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c28ff9640 1 -- 172.21.15.17:0/181485187 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c200abdb0 con 0x7f0c2c0b40f0 2024-03-16T10:11:38.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c32271640 1 -- 172.21.15.17:0/181485187 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2c0b1820 con 0x7f0c2c0b40f0 2024-03-16T10:11:38.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c28ff9640 1 -- 172.21.15.17:0/181485187 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c200b2070 con 0x7f0c2c0b40f0 2024-03-16T10:11:38.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c32271640 1 -- 172.21.15.17:0/181485187 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c2c0aff90 con 0x7f0c2c0b40f0 2024-03-16T10:11:38.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c32271640 1 -- 172.21.15.17:0/181485187 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c2c0b2f80 con 0x7f0c2c0b40f0 2024-03-16T10:11:38.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c28ff9640 1 -- 172.21.15.17:0/181485187 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f0c200c0070 con 0x7f0c2c0b40f0 2024-03-16T10:11:38.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c28ff9640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0c18067660 0x7f0c18069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:38.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c28ff9640 1 -- 172.21.15.17:0/181485187 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5227+0+0 (secure 0 0 0) 0x7f0c2011f7d0 con 0x7f0c2c0b40f0 2024-03-16T10:11:38.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.176+0000 7f0c2bfff640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0c18067660 0x7f0c18069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:38.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.180+0000 7f0c2bfff640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0c18067660 0x7f0c18069b20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c144c60 tx=0x7f0c1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:38.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.180+0000 7f0c28ff9640 1 -- 172.21.15.17:0/181485187 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0c2c0b2f80 con 0x7f0c2c0b40f0 2024-03-16T10:11:38.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:38.348+0000 7f0c32271640 1 -- 172.21.15.17:0/181485187 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0c2c06a130 con 0x7f0c2c0b40f0 2024-03-16T10:11:39.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.012+0000 7f0c28ff9640 1 -- 172.21.15.17:0/181485187 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v28) ==== 170+0+0 (secure 0 0 0) 0x7f0c200160d0 con 0x7f0c2c0b40f0 2024-03-16T10:11:39.017 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-03-16T10:11:39.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.016+0000 7f0c32271640 1 -- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0c18067660 msgr2=0x7f0c18069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:39.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.016+0000 7f0c32271640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0c18067660 0x7f0c18069b20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c144c60 tx=0x7f0c1c009040 comp rx=0 tx=0).stop 2024-03-16T10:11:39.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.016+0000 7f0c32271640 1 -- 172.21.15.17:0/181485187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c2c0b40f0 msgr2=0x7f0c2c0afa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:39.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.016+0000 7f0c32271640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c2c0b40f0 0x7f0c2c0afa50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0c200b0040 tx=0x7f0c20019130 comp rx=0 tx=0).stop 2024-03-16T10:11:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.016+0000 7f0c32271640 1 -- 172.21.15.17:0/181485187 shutdown_connections 2024-03-16T10:11:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.016+0000 7f0c32271640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0c18067660 0x7f0c18069b20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.016+0000 7f0c32271640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f0c2c0b5920 0x7f0c2c0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.016+0000 7f0c32271640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c2c0b40f0 0x7f0c2c0afa50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.016+0000 7f0c32271640 1 --2- 172.21.15.17:0/181485187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c2c067220 0x7f0c2c0af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.016+0000 7f0c32271640 1 -- 172.21.15.17:0/181485187 >> 172.21.15.17:0/181485187 conn(0x7f0c2c06b290 msgr2=0x7f0c2c06d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.016+0000 7f0c32271640 1 -- 172.21.15.17:0/181485187 shutdown_connections 2024-03-16T10:11:39.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.016+0000 7f0c32271640 1 -- 172.21.15.17:0/181485187 wait complete. 2024-03-16T10:11:39.057 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-16T10:11:39.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.176+0000 7fd42a824640 1 Processor -- start 2024-03-16T10:11:39.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.176+0000 7fd42a824640 1 -- start start 2024-03-16T10:11:39.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.176+0000 7fd42a824640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd4240b4200 0x7fd424150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:39.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.176+0000 7fd42a824640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4240af0c0 con 0x7fd4240b2400 2024-03-16T10:11:39.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.176+0000 7fd42a824640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4240b27e0 con 0x7fd4240ae060 2024-03-16T10:11:39.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.176+0000 7fd42a824640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd4240afcf0 con 0x7fd4240b4200 2024-03-16T10:11:39.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.176+0000 7fd428dc1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd4240b4200 0x7fd424150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:39.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.176+0000 7fd423fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd4240b2400 0x7fd4241452c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48394/0 (socket says 172.21.15.17:48394) 2024-03-16T10:11:39.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.176+0000 7fd428dc1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd4240b4200 0x7fd424150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52334/0 (socket says 172.21.15.17:52334) 2024-03-16T10:11:39.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.176+0000 7fd423fff640 1 -- 172.21.15.17:0/3020758564 learned_addr learned my addr 172.21.15.17:0/3020758564 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:39.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.176+0000 7fd428dc1640 1 -- 172.21.15.17:0/3020758564 >> v1:172.21.15.174:6789/0 conn(0x7fd4240ae060 legacy=0x7fd424072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:39.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.176+0000 7fd428dc1640 1 -- 172.21.15.17:0/3020758564 >> v1:172.21.15.17:6789/0 conn(0x7fd4240b2400 legacy=0x7fd4241452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:39.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.176+0000 7fd428dc1640 1 -- 172.21.15.17:0/3020758564 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4240af980 con 0x7fd4240b4200 2024-03-16T10:11:39.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd428dc1640 1 --2- 172.21.15.17:0/3020758564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4240b4200 0x7fd424150bd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009d90 tx=0x7fd41c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0a0e774c4fc1e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:39.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd422ffd640 1 -- 172.21.15.17:0/3020758564 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0b2070 con 0x7fd4240b4200 2024-03-16T10:11:39.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd422ffd640 1 -- 172.21.15.17:0/3020758564 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41c0a5940 con 0x7fd4240b4200 2024-03-16T10:11:39.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd422ffd640 1 -- 172.21.15.17:0/3020758564 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0a5d10 con 0x7fd4240b4200 2024-03-16T10:11:39.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 -- 172.21.15.17:0/3020758564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4240b4200 msgr2=0x7fd424150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:39.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 --2- 172.21.15.17:0/3020758564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4240b4200 0x7fd424150bd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009d90 tx=0x7fd41c01c970 comp rx=0 tx=0).stop 2024-03-16T10:11:39.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 -- 172.21.15.17:0/3020758564 shutdown_connections 2024-03-16T10:11:39.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 --2- 172.21.15.17:0/3020758564 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4240b4200 0x7fd424150bd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:39.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 -- 172.21.15.17:0/3020758564 >> 172.21.15.17:0/3020758564 conn(0x7fd4240b8e90 msgr2=0x7fd4240b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:39.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 -- 172.21.15.17:0/3020758564 shutdown_connections 2024-03-16T10:11:39.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 -- 172.21.15.17:0/3020758564 wait complete. 2024-03-16T10:11:39.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 Processor -- start 2024-03-16T10:11:39.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 -- start start 2024-03-16T10:11:39.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4240ae060 0x7fd424132790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:39.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4240b2400 0x7fd4241e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4240b4200 0x7fd424136bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd424153930 con 0x7fd4240b4200 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd4240b3fe0 con 0x7fd4240b2400 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd424153ab0 con 0x7fd4240ae060 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd428dc1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4240b4200 0x7fd424136bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd428dc1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4240b4200 0x7fd424136bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52338/0 (socket says 172.21.15.17:52338) 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd428dc1640 1 -- 172.21.15.17:0/3678569399 learned_addr learned my addr 172.21.15.17:0/3678569399 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd423fff640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4240ae060 0x7fd424132790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd4237fe640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4240b2400 0x7fd4241e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd428dc1640 1 -- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4240ae060 msgr2=0x7fd424132790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd428dc1640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4240ae060 0x7fd424132790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd428dc1640 1 -- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4240b2400 msgr2=0x7fd4241e35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd428dc1640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4240b2400 0x7fd4241e35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd428dc1640 1 -- 172.21.15.17:0/3678569399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4241360c0 con 0x7fd4240b4200 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd4237fe640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4240b2400 0x7fd4241e35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:39.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd428dc1640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4240b4200 0x7fd424136bc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0b0040 tx=0x7fd41c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:39.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd420ff9640 1 -- 172.21.15.17:0/3678569399 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0b2050 con 0x7fd4240b4200 2024-03-16T10:11:39.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd420ff9640 1 -- 172.21.15.17:0/3678569399 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd41c0be070 con 0x7fd4240b4200 2024-03-16T10:11:39.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd42a824640 1 -- 172.21.15.17:0/3678569399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd424134540 con 0x7fd4240b4200 2024-03-16T10:11:39.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.180+0000 7fd420ff9640 1 -- 172.21.15.17:0/3678569399 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0adaf0 con 0x7fd4240b4200 2024-03-16T10:11:39.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.184+0000 7fd42a824640 1 -- 172.21.15.17:0/3678569399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4241e3f60 con 0x7fd4240b4200 2024-03-16T10:11:39.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.184+0000 7fd42a824640 1 -- 172.21.15.17:0/3678569399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd424068360 con 0x7fd4240b4200 2024-03-16T10:11:39.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.184+0000 7fd420ff9640 1 -- 172.21.15.17:0/3678569399 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd41c0adc90 con 0x7fd4240b4200 2024-03-16T10:11:39.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.184+0000 7fd420ff9640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd404067690 0x7fd404069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:39.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.184+0000 7fd420ff9640 1 -- 172.21.15.17:0/3678569399 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5609+0+0 (secure 0 0 0) 0x7fd41c1204b0 con 0x7fd4240b4200 2024-03-16T10:11:39.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.184+0000 7fd423fff640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd404067690 0x7fd404069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:39.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.184+0000 7fd423fff640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd404067690 0x7fd404069b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd414005890 tx=0x7fd414009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:39.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.188+0000 7fd420ff9640 1 -- 172.21.15.17:0/3678569399 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd41c0c7020 con 0x7fd4240b4200 2024-03-16T10:11:39.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:39.384+0000 7fd42a824640 1 -- 172.21.15.17:0/3678569399 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fd42406a130 con 0x7fd4240b4200 2024-03-16T10:11:40.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.012+0000 7fd420ff9640 1 -- 172.21.15.17:0/3678569399 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fd41c016560 con 0x7fd4240b4200 2024-03-16T10:11:40.020 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 4 and data pool 5 2024-03-16T10:11:40.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.020+0000 7fd42a824640 1 -- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd404067690 msgr2=0x7fd404069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.020+0000 7fd42a824640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd404067690 0x7fd404069b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd414005890 tx=0x7fd414009040 comp rx=0 tx=0).stop 2024-03-16T10:11:40.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.020+0000 7fd42a824640 1 -- 172.21.15.17:0/3678569399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4240b4200 msgr2=0x7fd424136bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.020+0000 7fd42a824640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4240b4200 0x7fd424136bc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd41c0b0040 tx=0x7fd41c019710 comp rx=0 tx=0).stop 2024-03-16T10:11:40.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.020+0000 7fd42a824640 1 -- 172.21.15.17:0/3678569399 shutdown_connections 2024-03-16T10:11:40.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.020+0000 7fd42a824640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd404067690 0x7fd404069b50 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd414005890 tx=0x7fd414009040 comp rx=0 tx=0).stop 2024-03-16T10:11:40.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.020+0000 7fd42a824640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4240b4200 0x7fd424136bc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.020+0000 7fd42a824640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd4240b2400 0x7fd4241e35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.020+0000 7fd42a824640 1 --2- 172.21.15.17:0/3678569399 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd4240ae060 0x7fd424132790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.020+0000 7fd42a824640 1 -- 172.21.15.17:0/3678569399 >> 172.21.15.17:0/3678569399 conn(0x7fd4240b8e90 msgr2=0x7fd4240b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:40.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.024+0000 7fd42a824640 1 -- 172.21.15.17:0/3678569399 shutdown_connections 2024-03-16T10:11:40.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.024+0000 7fd42a824640 1 -- 172.21.15.17:0/3678569399 wait complete. 2024-03-16T10:11:40.075 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:40.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.184+0000 7f4233882640 1 Processor -- start 2024-03-16T10:11:40.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.184+0000 7f4233882640 1 -- start start 2024-03-16T10:11:40.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4233882640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f422c06d750 0x7f422c06f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4233882640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f422c145280 0x7f422c13f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4233882640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f422c06db30 con 0x7f422c0b2820 2024-03-16T10:11:40.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4233882640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f422c0b2d30 con 0x7f422c145280 2024-03-16T10:11:40.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4233882640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f422c06fcf0 con 0x7f422c06d750 2024-03-16T10:11:40.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4231e1f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f422c145280 0x7f422c13f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4230e1d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f422c0b2820 0x7f422c13eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48398/0 (socket says 172.21.15.17:48398) 2024-03-16T10:11:40.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4230e1d640 1 -- 172.21.15.17:0/3859092337 learned_addr learned my addr 172.21.15.17:0/3859092337 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:40.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f423161e640 1 --2- 172.21.15.17:0/3859092337 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f422c06d750 0x7f422c06f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4231e1f640 1 -- 172.21.15.17:0/3859092337 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f422c06d750 msgr2=0x7f422c06f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:40.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4231e1f640 1 --2- 172.21.15.17:0/3859092337 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f422c06d750 0x7f422c06f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4231e1f640 1 -- 172.21.15.17:0/3859092337 >> v1:172.21.15.17:6789/0 conn(0x7f422c0b2820 legacy=0x7f422c13eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4231e1f640 1 -- 172.21.15.17:0/3859092337 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422c0afaf0 con 0x7f422c145280 2024-03-16T10:11:40.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4231e1f640 1 --2- 172.21.15.17:0/3859092337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c145280 0x7f422c13f5f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f42240097f0 tx=0x7f422401c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca29cbf1d37bd4b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:40.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f421ffff640 1 -- 172.21.15.17:0/3859092337 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42240b0070 con 0x7f422c145280 2024-03-16T10:11:40.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f421ffff640 1 -- 172.21.15.17:0/3859092337 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42240ab600 con 0x7f422c145280 2024-03-16T10:11:40.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f421ffff640 1 -- 172.21.15.17:0/3859092337 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42240ab9d0 con 0x7f422c145280 2024-03-16T10:11:40.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4233882640 1 -- 172.21.15.17:0/3859092337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c145280 msgr2=0x7f422c13f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4233882640 1 --2- 172.21.15.17:0/3859092337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c145280 0x7f422c13f5f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f42240097f0 tx=0x7f422401c380 comp rx=0 tx=0).stop 2024-03-16T10:11:40.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4233882640 1 -- 172.21.15.17:0/3859092337 shutdown_connections 2024-03-16T10:11:40.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4233882640 1 --2- 172.21.15.17:0/3859092337 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c145280 0x7f422c13f5f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4233882640 1 --2- 172.21.15.17:0/3859092337 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f422c06d750 0x7f422c06f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.188+0000 7f4233882640 1 -- 172.21.15.17:0/3859092337 >> 172.21.15.17:0/3859092337 conn(0x7f422c0b55c0 msgr2=0x7f422c0b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:40.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4233882640 1 -- 172.21.15.17:0/3859092337 shutdown_connections 2024-03-16T10:11:40.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4233882640 1 -- 172.21.15.17:0/3859092337 wait complete. 2024-03-16T10:11:40.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4233882640 1 Processor -- start 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4233882640 1 -- start start 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4233882640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f422c06d750 0x7f422c13bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4233882640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c0b2820 0x7f422c1e3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4233882640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f422c145280 0x7f422c1e3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4233882640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f422c152550 con 0x7f422c0b2820 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4233882640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f422c0b49f0 con 0x7f422c06d750 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4233882640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f422c1526d0 con 0x7f422c145280 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4230e1d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c0b2820 0x7f422c1e3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4230e1d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c0b2820 0x7f422c1e3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52352/0 (socket says 172.21.15.17:52352) 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4230e1d640 1 -- 172.21.15.17:0/85521507 learned_addr learned my addr 172.21.15.17:0/85521507 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f423161e640 1 --2- 172.21.15.17:0/85521507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f422c06d750 0x7f422c13bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4230e1d640 1 -- 172.21.15.17:0/85521507 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f422c145280 msgr2=0x7f422c1e3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4231e1f640 1 --2- 172.21.15.17:0/85521507 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f422c145280 0x7f422c1e3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4230e1d640 1 --2- 172.21.15.17:0/85521507 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f422c145280 0x7f422c1e3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4230e1d640 1 -- 172.21.15.17:0/85521507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f422c06d750 msgr2=0x7f422c13bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4230e1d640 1 --2- 172.21.15.17:0/85521507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f422c06d750 0x7f422c13bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4230e1d640 1 -- 172.21.15.17:0/85521507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422c13c3f0 con 0x7f422c0b2820 2024-03-16T10:11:40.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4230e1d640 1 --2- 172.21.15.17:0/85521507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c0b2820 0x7f422c1e3130 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f422000cae0 tx=0x7f422000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:40.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f421dffb640 1 -- 172.21.15.17:0/85521507 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220017070 con 0x7f422c0b2820 2024-03-16T10:11:40.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f421dffb640 1 -- 172.21.15.17:0/85521507 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f422000ac40 con 0x7f422c0b2820 2024-03-16T10:11:40.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4233882640 1 -- 172.21.15.17:0/85521507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f422c13a480 con 0x7f422c0b2820 2024-03-16T10:11:40.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f421dffb640 1 -- 172.21.15.17:0/85521507 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4220012620 con 0x7f422c0b2820 2024-03-16T10:11:40.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4233882640 1 -- 172.21.15.17:0/85521507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f422c0b1ed0 con 0x7f422c0b2820 2024-03-16T10:11:40.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.192+0000 7f4233882640 1 -- 172.21.15.17:0/85521507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f422c068360 con 0x7f422c0b2820 2024-03-16T10:11:40.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.196+0000 7f421dffb640 1 -- 172.21.15.17:0/85521507 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f422001a070 con 0x7f422c0b2820 2024-03-16T10:11:40.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.196+0000 7f421dffb640 1 --2- 172.21.15.17:0/85521507 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4210067660 0x7f4210069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.196+0000 7f421dffb640 1 -- 172.21.15.17:0/85521507 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f42200855f0 con 0x7f422c0b2820 2024-03-16T10:11:40.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.196+0000 7f423161e640 1 --2- 172.21.15.17:0/85521507 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4210067660 0x7f4210069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.196+0000 7f423161e640 1 --2- 172.21.15.17:0/85521507 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4210067660 0x7f4210069b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f422800b9d0 tx=0x7f4228008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:40.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.200+0000 7f421dffb640 1 -- 172.21.15.17:0/85521507 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f4220056300 con 0x7f422c0b2820 2024-03-16T10:11:40.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.360+0000 7f4233882640 1 -- 172.21.15.17:0/85521507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f422c142ed0 con 0x7f422c0b2820 2024-03-16T10:11:40.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.364+0000 7f421dffb640 1 -- 172.21.15.17:0/85521507 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16682 (secure 0 0 0) 0x7f4220055c00 con 0x7f422c0b2820 2024-03-16T10:11:40.368 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:40.368 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":29,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:11:39.392000+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:11:37.284106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T10:11:38.392554+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.364+0000 7f4233882640 1 -- 172.21.15.17:0/85521507 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4210067660 msgr2=0x7f4210069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.364+0000 7f4233882640 1 --2- 172.21.15.17:0/85521507 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4210067660 0x7f4210069b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f422800b9d0 tx=0x7f4228008040 comp rx=0 tx=0).stop 2024-03-16T10:11:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.364+0000 7f4233882640 1 -- 172.21.15.17:0/85521507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c0b2820 msgr2=0x7f422c1e3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.364+0000 7f4233882640 1 --2- 172.21.15.17:0/85521507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c0b2820 0x7f422c1e3130 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f422000cae0 tx=0x7f422000a060 comp rx=0 tx=0).stop 2024-03-16T10:11:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.364+0000 7f4233882640 1 -- 172.21.15.17:0/85521507 shutdown_connections 2024-03-16T10:11:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.364+0000 7f4233882640 1 --2- 172.21.15.17:0/85521507 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4210067660 0x7f4210069b20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.364+0000 7f4233882640 1 --2- 172.21.15.17:0/85521507 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f422c145280 0x7f422c1e3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.364+0000 7f4233882640 1 --2- 172.21.15.17:0/85521507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f422c0b2820 0x7f422c1e3130 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.364+0000 7f4233882640 1 --2- 172.21.15.17:0/85521507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f422c06d750 0x7f422c13bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.364+0000 7f4233882640 1 -- 172.21.15.17:0/85521507 >> 172.21.15.17:0/85521507 conn(0x7f422c0b55c0 msgr2=0x7f422c1431f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:40.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.364+0000 7f4233882640 1 -- 172.21.15.17:0/85521507 shutdown_connections 2024-03-16T10:11:40.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.364+0000 7f4233882640 1 -- 172.21.15.17:0/85521507 wait complete. 2024-03-16T10:11:40.385 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:40.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe544144640 1 Processor -- start 2024-03-16T10:11:40.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe544144640 1 -- start start 2024-03-16T10:11:40.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe544144640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe53c0b5710 0x7fe53c0b0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe544144640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53c06e720 0x7fe53c147460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe544144640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe53c09f600 con 0x7fe53c06de30 2024-03-16T10:11:40.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe544144640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fe53c09fde0 con 0x7fe53c0b5710 2024-03-16T10:11:40.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe544144640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c073390 con 0x7fe53c06e720 2024-03-16T10:11:40.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe5426e1640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe53c06de30 0x7fe53c06b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48404/0 (socket says 172.21.15.17:48404) 2024-03-16T10:11:40.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe5426e1640 1 -- 172.21.15.17:0/4073582505 learned_addr learned my addr 172.21.15.17:0/4073582505 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:40.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe541ee0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fe53c0b5710 0x7fe53c0b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe5416df640 1 --2- 172.21.15.17:0/4073582505 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53c06e720 0x7fe53c147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe540ede640 1 -- 172.21.15.17:0/4073582505 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1998069653 0 0) 0x7fe53c09f600 con 0x7fe53c06de30 2024-03-16T10:11:40.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe540ede640 1 -- 172.21.15.17:0/4073582505 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe528003530 con 0x7fe53c06de30 2024-03-16T10:11:40.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe541ee0640 1 -- 172.21.15.17:0/4073582505 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53c06e720 msgr2=0x7fe53c147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:40.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe541ee0640 1 --2- 172.21.15.17:0/4073582505 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53c06e720 0x7fe53c147460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe541ee0640 1 -- 172.21.15.17:0/4073582505 >> v1:172.21.15.17:6789/0 conn(0x7fe53c06de30 legacy=0x7fe53c06b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe541ee0640 1 -- 172.21.15.17:0/4073582505 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c09f8c0 con 0x7fe53c0b5710 2024-03-16T10:11:40.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.496+0000 7fe5416df640 1 --2- 172.21.15.17:0/4073582505 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53c06e720 0x7fe53c147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:40.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe541ee0640 1 --2- 172.21.15.17:0/4073582505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe53c0b5710 0x7fe53c0b0790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0097c0 tx=0x7fe52c01ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdf48c970cdead4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:40.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe540ede640 1 -- 172.21.15.17:0/4073582505 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0b0070 con 0x7fe53c0b5710 2024-03-16T10:11:40.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe540ede640 1 -- 172.21.15.17:0/4073582505 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe52c0ab600 con 0x7fe53c0b5710 2024-03-16T10:11:40.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe540ede640 1 -- 172.21.15.17:0/4073582505 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0ab9b0 con 0x7fe53c0b5710 2024-03-16T10:11:40.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 -- 172.21.15.17:0/4073582505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe53c0b5710 msgr2=0x7fe53c0b0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 --2- 172.21.15.17:0/4073582505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe53c0b5710 0x7fe53c0b0790 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0097c0 tx=0x7fe52c01ce70 comp rx=0 tx=0).stop 2024-03-16T10:11:40.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 -- 172.21.15.17:0/4073582505 shutdown_connections 2024-03-16T10:11:40.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 --2- 172.21.15.17:0/4073582505 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53c06e720 0x7fe53c147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 --2- 172.21.15.17:0/4073582505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe53c0b5710 0x7fe53c0b0790 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 -- 172.21.15.17:0/4073582505 >> 172.21.15.17:0/4073582505 conn(0x7fe53c0b6800 msgr2=0x7fe53c0b6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:40.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 -- 172.21.15.17:0/4073582505 shutdown_connections 2024-03-16T10:11:40.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 -- 172.21.15.17:0/4073582505 wait complete. 2024-03-16T10:11:40.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 Processor -- start 2024-03-16T10:11:40.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 -- start start 2024-03-16T10:11:40.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe53c0b5710 0x7fe53c144980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe53c143a60 0x7fe53c1426c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53c142c00 0x7fe53c136e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c147d30 con 0x7fe53c143a60 2024-03-16T10:11:40.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c147bb0 con 0x7fe53c0b5710 2024-03-16T10:11:40.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe544144640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe53c154de0 con 0x7fe53c142c00 2024-03-16T10:11:40.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe5416df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe53c143a60 0x7fe53c1426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe5416df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe53c143a60 0x7fe53c1426c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52366/0 (socket says 172.21.15.17:52366) 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.500+0000 7fe5416df640 1 -- 172.21.15.17:0/1861690109 learned_addr learned my addr 172.21.15.17:0/1861690109 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe5426e1640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53c142c00 0x7fe53c136e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe541ee0640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe53c0b5710 0x7fe53c144980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe5416df640 1 -- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53c142c00 msgr2=0x7fe53c136e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe5416df640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53c142c00 0x7fe53c136e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe5416df640 1 -- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe53c0b5710 msgr2=0x7fe53c144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe5416df640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe53c0b5710 0x7fe53c144980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe5416df640 1 -- 172.21.15.17:0/1861690109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c143090 con 0x7fe53c143a60 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe541ee0640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe53c0b5710 0x7fe53c144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe5416df640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe53c143a60 0x7fe53c1426c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe53800e840 tx=0x7fe53800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe5327fc640 1 -- 172.21.15.17:0/1861690109 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe538017070 con 0x7fe53c143a60 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe5327fc640 1 -- 172.21.15.17:0/1861690109 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53800c8d0 con 0x7fe53c143a60 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe544144640 1 -- 172.21.15.17:0/1861690109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53c143f40 con 0x7fe53c143a60 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe5327fc640 1 -- 172.21.15.17:0/1861690109 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53800f040 con 0x7fe53c143a60 2024-03-16T10:11:40.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe544144640 1 -- 172.21.15.17:0/1861690109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53c144220 con 0x7fe53c143a60 2024-03-16T10:11:40.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe544144640 1 -- 172.21.15.17:0/1861690109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe53c06a130 con 0x7fe53c143a60 2024-03-16T10:11:40.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe5327fc640 1 -- 172.21.15.17:0/1861690109 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe53801a070 con 0x7fe53c143a60 2024-03-16T10:11:40.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe5327fc640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe520067660 0x7fe520069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe5327fc640 1 -- 172.21.15.17:0/1861690109 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fe538085400 con 0x7fe53c143a60 2024-03-16T10:11:40.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.504+0000 7fe541ee0640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe520067660 0x7fe520069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.508+0000 7fe541ee0640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe520067660 0x7fe520069b20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0ad000 tx=0x7fe52c004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:40.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.508+0000 7fe5327fc640 1 -- 172.21.15.17:0/1861690109 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe53c143f40 con 0x7fe53c143a60 2024-03-16T10:11:40.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.672+0000 7fe544144640 1 -- 172.21.15.17:0/1861690109 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe53c147020 con 0x7fe53c143a60 2024-03-16T10:11:40.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.676+0000 7fe5327fc640 1 -- 172.21.15.17:0/1861690109 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16682 (secure 0 0 0) 0x7fe538056110 con 0x7fe53c143a60 2024-03-16T10:11:40.680 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:40.680 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":29,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:11:39.392000+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:11:37.284106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T10:11:38.392554+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:40.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.676+0000 7fe544144640 1 -- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe520067660 msgr2=0x7fe520069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.676+0000 7fe544144640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe520067660 0x7fe520069b20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0ad000 tx=0x7fe52c004740 comp rx=0 tx=0).stop 2024-03-16T10:11:40.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.676+0000 7fe544144640 1 -- 172.21.15.17:0/1861690109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe53c143a60 msgr2=0x7fe53c1426c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.676+0000 7fe544144640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe53c143a60 0x7fe53c1426c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe53800e840 tx=0x7fe53800ed00 comp rx=0 tx=0).stop 2024-03-16T10:11:40.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.676+0000 7fe544144640 1 -- 172.21.15.17:0/1861690109 shutdown_connections 2024-03-16T10:11:40.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.676+0000 7fe544144640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe520067660 0x7fe520069b20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.676+0000 7fe544144640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe53c142c00 0x7fe53c136e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.676+0000 7fe544144640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe53c143a60 0x7fe53c1426c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.676+0000 7fe544144640 1 --2- 172.21.15.17:0/1861690109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe53c0b5710 0x7fe53c144980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.676+0000 7fe544144640 1 -- 172.21.15.17:0/1861690109 >> 172.21.15.17:0/1861690109 conn(0x7fe53c0b6800 msgr2=0x7fe53c0b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:40.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.676+0000 7fe544144640 1 -- 172.21.15.17:0/1861690109 shutdown_connections 2024-03-16T10:11:40.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.676+0000 7fe544144640 1 -- 172.21.15.17:0/1861690109 wait complete. 2024-03-16T10:11:40.699 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-16T10:11:40.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.816+0000 7fab0ec4a640 1 Processor -- start 2024-03-16T10:11:40.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.816+0000 7fab0ec4a640 1 -- start start 2024-03-16T10:11:40.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.820+0000 7fab0ec4a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fab080b4650 0x7fab0814b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.820+0000 7fab0ec4a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab080b4a30 con 0x7fab081419f0 2024-03-16T10:11:40.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.820+0000 7fab0ec4a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab0814abe0 con 0x7fab080b7650 2024-03-16T10:11:40.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.820+0000 7fab0ec4a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fab0814aec0 con 0x7fab080b4650 2024-03-16T10:11:40.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.820+0000 7fab0c9e6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fab080b4650 0x7fab0814b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.820+0000 7fab0c9e6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fab080b4650 0x7fab0814b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52368/0 (socket says 172.21.15.17:52368) 2024-03-16T10:11:40.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.820+0000 7fab0c9e6640 1 -- 172.21.15.17:0/3071710803 learned_addr learned my addr 172.21.15.17:0/3071710803 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:40.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.820+0000 7fab0c9e6640 1 -- 172.21.15.17:0/3071710803 >> v1:172.21.15.174:6789/0 conn(0x7fab080b7650 legacy=0x7fab081412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.820+0000 7fab0c9e6640 1 -- 172.21.15.17:0/3071710803 >> v1:172.21.15.17:6789/0 conn(0x7fab081419f0 legacy=0x7fab08151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.820+0000 7fab0c9e6640 1 -- 172.21.15.17:0/3071710803 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab0814b920 con 0x7fab080b4650 2024-03-16T10:11:40.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.820+0000 7fab0c9e6640 1 --2- 172.21.15.17:0/3071710803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab080b4650 0x7fab0814b3e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009870 tx=0x7faaf801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244265b6f53eb464 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:40.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.820+0000 7fab077fe640 1 -- 172.21.15.17:0/3071710803 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf80b0070 con 0x7fab080b4650 2024-03-16T10:11:40.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.820+0000 7fab077fe640 1 -- 172.21.15.17:0/3071710803 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaf80ab600 con 0x7fab080b4650 2024-03-16T10:11:40.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.820+0000 7fab077fe640 1 -- 172.21.15.17:0/3071710803 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf80ab9d0 con 0x7fab080b4650 2024-03-16T10:11:40.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3071710803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab080b4650 msgr2=0x7fab0814b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 --2- 172.21.15.17:0/3071710803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab080b4650 0x7fab0814b3e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009870 tx=0x7faaf801c4d0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3071710803 shutdown_connections 2024-03-16T10:11:40.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 --2- 172.21.15.17:0/3071710803 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab080b4650 0x7fab0814b3e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3071710803 >> 172.21.15.17:0/3071710803 conn(0x7fab080bccd0 msgr2=0x7fab080bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:40.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3071710803 shutdown_connections 2024-03-16T10:11:40.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3071710803 wait complete. 2024-03-16T10:11:40.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 Processor -- start 2024-03-16T10:11:40.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 -- start start 2024-03-16T10:11:40.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab080b4650 0x7fab08142ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab080b7650 0x7fab081439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab081419f0 0x7fab081e0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fab080b5b70 con 0x7fab080b7650 2024-03-16T10:11:40.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fab080b59f0 con 0x7fab081419f0 2024-03-16T10:11:40.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fab08155d10 con 0x7fab080b4650 2024-03-16T10:11:40.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab07fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab080b7650 0x7fab081439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab07fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab080b7650 0x7fab081439b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52376/0 (socket says 172.21.15.17:52376) 2024-03-16T10:11:40.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab07fff640 1 -- 172.21.15.17:0/3702182942 learned_addr learned my addr 172.21.15.17:0/3702182942 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:40.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0c9e6640 1 --2- 172.21.15.17:0/3702182942 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab080b4650 0x7fab08142ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0d1e7640 1 --2- 172.21.15.17:0/3702182942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab081419f0 0x7fab081e0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab07fff640 1 -- 172.21.15.17:0/3702182942 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab080b4650 msgr2=0x7fab08142ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab07fff640 1 --2- 172.21.15.17:0/3702182942 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab080b4650 0x7fab08142ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab07fff640 1 -- 172.21.15.17:0/3702182942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab081419f0 msgr2=0x7fab081e0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:40.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab07fff640 1 --2- 172.21.15.17:0/3702182942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab081419f0 0x7fab081e0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:40.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab07fff640 1 -- 172.21.15.17:0/3702182942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab081444c0 con 0x7fab080b7650 2024-03-16T10:11:40.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab07fff640 1 --2- 172.21.15.17:0/3702182942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab080b7650 0x7fab081439b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faafc00c420 tx=0x7faafc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:40.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab057fa640 1 -- 172.21.15.17:0/3702182942 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc0052d0 con 0x7fab080b7650 2024-03-16T10:11:40.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab057fa640 1 -- 172.21.15.17:0/3702182942 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faafc005470 con 0x7fab080b7650 2024-03-16T10:11:40.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab057fa640 1 -- 172.21.15.17:0/3702182942 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faafc007260 con 0x7fab080b7650 2024-03-16T10:11:40.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3702182942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab08144c30 con 0x7fab080b7650 2024-03-16T10:11:40.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.824+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3702182942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab081440c0 con 0x7fab080b7650 2024-03-16T10:11:40.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.828+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3702182942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab08068360 con 0x7fab080b7650 2024-03-16T10:11:40.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.828+0000 7fab057fa640 1 -- 172.21.15.17:0/3702182942 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7faafc007400 con 0x7fab080b7650 2024-03-16T10:11:40.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.828+0000 7fab057fa640 1 --2- 172.21.15.17:0/3702182942 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7faaf4067660 0x7faaf4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:40.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.828+0000 7fab057fa640 1 -- 172.21.15.17:0/3702182942 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7faafc084df0 con 0x7fab080b7650 2024-03-16T10:11:40.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.828+0000 7fab0c9e6640 1 --2- 172.21.15.17:0/3702182942 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7faaf4067660 0x7faaf4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:40.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.828+0000 7fab0c9e6640 1 --2- 172.21.15.17:0/3702182942 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7faaf4067660 0x7faaf4069b20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faaf80ad000 tx=0x7faaf8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:40.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:40.832+0000 7fab057fa640 1 -- 172.21.15.17:0/3702182942 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7faafc055a60 con 0x7fab080b7650 2024-03-16T10:11:41.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.032+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3702182942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fab08152d60 con 0x7fab080b7650 2024-03-16T10:11:41.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.044+0000 7fab057fa640 1 -- 172.21.15.17:0/3702182942 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) ==== 118+0+0 (secure 0 0 0) 0x7faafc055360 con 0x7fab080b7650 2024-03-16T10:11:41.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.048+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3702182942 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7faaf4067660 msgr2=0x7faaf4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:41.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.048+0000 7fab0ec4a640 1 --2- 172.21.15.17:0/3702182942 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7faaf4067660 0x7faaf4069b20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faaf80ad000 tx=0x7faaf8008040 comp rx=0 tx=0).stop 2024-03-16T10:11:41.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.048+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3702182942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab080b7650 msgr2=0x7fab081439b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:41.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.048+0000 7fab0ec4a640 1 --2- 172.21.15.17:0/3702182942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab080b7650 0x7fab081439b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faafc00c420 tx=0x7faafc00c8e0 comp rx=0 tx=0).stop 2024-03-16T10:11:41.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.048+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3702182942 shutdown_connections 2024-03-16T10:11:41.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.048+0000 7fab0ec4a640 1 --2- 172.21.15.17:0/3702182942 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7faaf4067660 0x7faaf4069b20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:41.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.048+0000 7fab0ec4a640 1 --2- 172.21.15.17:0/3702182942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab081419f0 0x7fab081e0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:41.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.048+0000 7fab0ec4a640 1 --2- 172.21.15.17:0/3702182942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fab080b7650 0x7fab081439b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:41.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.048+0000 7fab0ec4a640 1 --2- 172.21.15.17:0/3702182942 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fab080b4650 0x7fab08142ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:41.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.048+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3702182942 >> 172.21.15.17:0/3702182942 conn(0x7fab080bccd0 msgr2=0x7fab0813ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:41.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.048+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3702182942 shutdown_connections 2024-03-16T10:11:41.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.048+0000 7fab0ec4a640 1 -- 172.21.15.17:0/3702182942 wait complete. 2024-03-16T10:11:41.070 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-16T10:11:41.070 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-16T10:11:41.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec3415640 1 Processor -- start 2024-03-16T10:11:41.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec3415640 1 -- start start 2024-03-16T10:11:41.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec3415640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2ebc06e3c0 0x7f2ebc0af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:41.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec3415640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2ebc0af8d0 0x7f2ebc147d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:41.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec3415640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ebc06e7a0 con 0x7f2ebc1482a0 2024-03-16T10:11:41.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec3415640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f2ebc09f260 con 0x7f2ebc06e3c0 2024-03-16T10:11:41.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec3415640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f2ebc0aec20 con 0x7f2ebc0af8d0 2024-03-16T10:11:41.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec11b1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2ebc06e3c0 0x7f2ebc0af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:41.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec19b2640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2ebc1482a0 0x7f2ebc0b54e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48434/0 (socket says 172.21.15.17:48434) 2024-03-16T10:11:41.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec19b2640 1 -- 172.21.15.17:0/1800152980 learned_addr learned my addr 172.21.15.17:0/1800152980 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:41.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec09b0640 1 --2- 172.21.15.17:0/1800152980 >> v2:172.21.15.174:3300/0 conn(0x7f2ebc0af8d0 0x7f2ebc147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:41.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec11b1640 1 -- 172.21.15.17:0/1800152980 >> v2:172.21.15.174:3300/0 conn(0x7f2ebc0af8d0 msgr2=0x7f2ebc147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:41.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec11b1640 1 --2- 172.21.15.17:0/1800152980 >> v2:172.21.15.174:3300/0 conn(0x7f2ebc0af8d0 0x7f2ebc147d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:41.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec11b1640 1 -- 172.21.15.17:0/1800152980 >> v1:172.21.15.17:6789/0 conn(0x7f2ebc1482a0 legacy=0x7f2ebc0b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:41.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2ec11b1640 1 -- 172.21.15.17:0/1800152980 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ebc06dd10 con 0x7f2ebc06e3c0 2024-03-16T10:11:41.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.196+0000 7f2eb3fff640 1 -- 172.21.15.17:0/1800152980 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 288745734 0 0) 0x7f2ebc06e7a0 con 0x7f2ebc1482a0 2024-03-16T10:11:41.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec11b1640 1 --2- 172.21.15.17:0/1800152980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ebc06e3c0 0x7f2ebc0af390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc06d340 tx=0x7f2eb801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f984bd9d0fbeb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:41.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2eb3fff640 1 -- 172.21.15.17:0/1800152980 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80b0070 con 0x7f2ebc06e3c0 2024-03-16T10:11:41.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2eb3fff640 1 -- 172.21.15.17:0/1800152980 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb80ab600 con 0x7f2ebc06e3c0 2024-03-16T10:11:41.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2eb3fff640 1 -- 172.21.15.17:0/1800152980 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80ab9d0 con 0x7f2ebc06e3c0 2024-03-16T10:11:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 -- 172.21.15.17:0/1800152980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ebc06e3c0 msgr2=0x7f2ebc0af390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 --2- 172.21.15.17:0/1800152980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ebc06e3c0 0x7f2ebc0af390 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc06d340 tx=0x7f2eb801c480 comp rx=0 tx=0).stop 2024-03-16T10:11:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 -- 172.21.15.17:0/1800152980 shutdown_connections 2024-03-16T10:11:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 --2- 172.21.15.17:0/1800152980 >> v2:172.21.15.174:3300/0 conn(0x7f2ebc0af8d0 0x7f2ebc147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 --2- 172.21.15.17:0/1800152980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ebc06e3c0 0x7f2ebc0af390 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 -- 172.21.15.17:0/1800152980 >> 172.21.15.17:0/1800152980 conn(0x7f2ebc0b65d0 msgr2=0x7f2ebc0b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 -- 172.21.15.17:0/1800152980 shutdown_connections 2024-03-16T10:11:41.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 -- 172.21.15.17:0/1800152980 wait complete. 2024-03-16T10:11:41.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 Processor -- start 2024-03-16T10:11:41.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 -- start start 2024-03-16T10:11:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ebc06e3c0 0x7f2ebc138e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ebc0af8d0 0x7f2ebc136e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ebc1482a0 0x7f2ebc148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc06f070 con 0x7f2ebc0af8d0 2024-03-16T10:11:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ebc1486b0 con 0x7f2ebc1482a0 2024-03-16T10:11:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec3415640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2ebc06d4d0 con 0x7f2ebc06e3c0 2024-03-16T10:11:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec09b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ebc0af8d0 0x7f2ebc136e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec09b0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ebc0af8d0 0x7f2ebc136e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52404/0 (socket says 172.21.15.17:52404) 2024-03-16T10:11:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec09b0640 1 -- 172.21.15.17:0/1878564361 learned_addr learned my addr 172.21.15.17:0/1878564361 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec19b2640 1 --2- 172.21.15.17:0/1878564361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ebc1482a0 0x7f2ebc148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec11b1640 1 --2- 172.21.15.17:0/1878564361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ebc06e3c0 0x7f2ebc138e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:41.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec09b0640 1 -- 172.21.15.17:0/1878564361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ebc06e3c0 msgr2=0x7f2ebc138e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:41.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec09b0640 1 --2- 172.21.15.17:0/1878564361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ebc06e3c0 0x7f2ebc138e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:41.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec09b0640 1 -- 172.21.15.17:0/1878564361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ebc1482a0 msgr2=0x7f2ebc148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:41.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec09b0640 1 --2- 172.21.15.17:0/1878564361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ebc1482a0 0x7f2ebc148ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:41.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec09b0640 1 -- 172.21.15.17:0/1878564361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ebc14a7b0 con 0x7f2ebc0af8d0 2024-03-16T10:11:41.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2ec09b0640 1 --2- 172.21.15.17:0/1878564361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ebc0af8d0 0x7f2ebc136e80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00d800 tx=0x7f2eac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:41.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.200+0000 7f2eb1ffb640 1 -- 172.21.15.17:0/1878564361 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac00b6c0 con 0x7f2ebc0af8d0 2024-03-16T10:11:41.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.204+0000 7f2eb1ffb640 1 -- 172.21.15.17:0/1878564361 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eac019070 con 0x7f2ebc0af8d0 2024-03-16T10:11:41.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.204+0000 7f2ec3415640 1 -- 172.21.15.17:0/1878564361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ebc14aa90 con 0x7f2ebc0af8d0 2024-03-16T10:11:41.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.204+0000 7f2eb1ffb640 1 -- 172.21.15.17:0/1878564361 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac00ba60 con 0x7f2ebc0af8d0 2024-03-16T10:11:41.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.204+0000 7f2ec3415640 1 -- 172.21.15.17:0/1878564361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ebc14b430 con 0x7f2ebc0af8d0 2024-03-16T10:11:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.204+0000 7f2ec3415640 1 -- 172.21.15.17:0/1878564361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ebc14af80 con 0x7f2ebc0af8d0 2024-03-16T10:11:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.204+0000 7f2eb1ffb640 1 -- 172.21.15.17:0/1878564361 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f2eac00bc00 con 0x7f2ebc0af8d0 2024-03-16T10:11:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.204+0000 7f2eb1ffb640 1 --2- 172.21.15.17:0/1878564361 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2ea0067660 0x7f2ea0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.204+0000 7f2eb1ffb640 1 -- 172.21.15.17:0/1878564361 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f2eac085d00 con 0x7f2ebc0af8d0 2024-03-16T10:11:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.204+0000 7f2ec11b1640 1 --2- 172.21.15.17:0/1878564361 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2ea0067660 0x7f2ea0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:41.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.204+0000 7f2ec11b1640 1 --2- 172.21.15.17:0/1878564361 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2ea0067660 0x7f2ea0069b20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80ad000 tx=0x7f2eb80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:41.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.212+0000 7f2eb1ffb640 1 -- 172.21.15.17:0/1878564361 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2eac056990 con 0x7f2ebc0af8d0 2024-03-16T10:11:41.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:41.412+0000 7f2ec3415640 1 -- 172.21.15.17:0/1878564361 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2ebc06a130 con 0x7f2ebc0af8d0 2024-03-16T10:11:42.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.056+0000 7f2eb1ffb640 1 -- 172.21.15.17:0/1878564361 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) ==== 122+0+0 (secure 0 0 0) 0x7f2eac056290 con 0x7f2ebc0af8d0 2024-03-16T10:11:42.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.060+0000 7f2ec3415640 1 -- 172.21.15.17:0/1878564361 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2ea0067660 msgr2=0x7f2ea0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.060+0000 7f2ec3415640 1 --2- 172.21.15.17:0/1878564361 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2ea0067660 0x7f2ea0069b20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2eb80ad000 tx=0x7f2eb80a6000 comp rx=0 tx=0).stop 2024-03-16T10:11:42.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.060+0000 7f2ec3415640 1 -- 172.21.15.17:0/1878564361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ebc0af8d0 msgr2=0x7f2ebc136e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.060+0000 7f2ec3415640 1 --2- 172.21.15.17:0/1878564361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ebc0af8d0 0x7f2ebc136e80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2eac00d800 tx=0x7f2eac00dcc0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.060+0000 7f2ec3415640 1 -- 172.21.15.17:0/1878564361 shutdown_connections 2024-03-16T10:11:42.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.060+0000 7f2ec3415640 1 --2- 172.21.15.17:0/1878564361 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2ea0067660 0x7f2ea0069b20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.060+0000 7f2ec3415640 1 --2- 172.21.15.17:0/1878564361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ebc1482a0 0x7f2ebc148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.060+0000 7f2ec3415640 1 --2- 172.21.15.17:0/1878564361 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ebc0af8d0 0x7f2ebc136e80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.060+0000 7f2ec3415640 1 --2- 172.21.15.17:0/1878564361 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2ebc06e3c0 0x7f2ebc138e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.060+0000 7f2ec3415640 1 -- 172.21.15.17:0/1878564361 >> 172.21.15.17:0/1878564361 conn(0x7f2ebc0b65d0 msgr2=0x7f2ebc0b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:42.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.060+0000 7f2ec3415640 1 -- 172.21.15.17:0/1878564361 shutdown_connections 2024-03-16T10:11:42.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.060+0000 7f2ec3415640 1 -- 172.21.15.17:0/1878564361 wait complete. 2024-03-16T10:11:42.091 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-16T10:11:42.092 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:11:42.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.200+0000 7ffbb83d9640 1 Processor -- start 2024-03-16T10:11:42.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.200+0000 7ffbb83d9640 1 -- start start 2024-03-16T10:11:42.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.200+0000 7ffbb83d9640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ffbb0066fc0 0x7ffbb00af980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb83d9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ffbb013e2f0 0x7ffbb014eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb83d9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb00b0200 con 0x7ffbb00673a0 2024-03-16T10:11:42.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb83d9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb00b04e0 con 0x7ffbb013e2f0 2024-03-16T10:11:42.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb83d9640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb00b5020 con 0x7ffbb0066fc0 2024-03-16T10:11:42.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb6976640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ffbb013e2f0 0x7ffbb014eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb6976640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7ffbb013e2f0 0x7ffbb014eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52416/0 (socket says 172.21.15.17:52416) 2024-03-16T10:11:42.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb6976640 1 -- 172.21.15.17:0/2891102482 learned_addr learned my addr 172.21.15.17:0/2891102482 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:42.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb6175640 1 --2- 172.21.15.17:0/2891102482 >> v2:172.21.15.174:3300/0 conn(0x7ffbb0066fc0 0x7ffbb00af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb6976640 1 -- 172.21.15.17:0/2891102482 >> v2:172.21.15.174:3300/0 conn(0x7ffbb0066fc0 msgr2=0x7ffbb00af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:42.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb6976640 1 --2- 172.21.15.17:0/2891102482 >> v2:172.21.15.174:3300/0 conn(0x7ffbb0066fc0 0x7ffbb00af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb6976640 1 -- 172.21.15.17:0/2891102482 >> v1:172.21.15.17:6789/0 conn(0x7ffbb00673a0 legacy=0x7ffbb00b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb6976640 1 -- 172.21.15.17:0/2891102482 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb0070720 con 0x7ffbb013e2f0 2024-03-16T10:11:42.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb5173640 1 -- 172.21.15.17:0/2891102482 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 399925214 0 0) 0x7ffbb00b0200 con 0x7ffbb00673a0 2024-03-16T10:11:42.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb6976640 1 --2- 172.21.15.17:0/2891102482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb013e2f0 0x7ffbb014eb30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0067820 tx=0x7ffba801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1f2a03718357ae server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:42.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb5173640 1 -- 172.21.15.17:0/2891102482 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba80a5730 con 0x7ffbb013e2f0 2024-03-16T10:11:42.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb5173640 1 -- 172.21.15.17:0/2891102482 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba80a58d0 con 0x7ffbb013e2f0 2024-03-16T10:11:42.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb5173640 1 -- 172.21.15.17:0/2891102482 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba80a5ca0 con 0x7ffbb013e2f0 2024-03-16T10:11:42.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb83d9640 1 -- 172.21.15.17:0/2891102482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb013e2f0 msgr2=0x7ffbb014eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb83d9640 1 --2- 172.21.15.17:0/2891102482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb013e2f0 0x7ffbb014eb30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0067820 tx=0x7ffba801c960 comp rx=0 tx=0).stop 2024-03-16T10:11:42.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb83d9640 1 -- 172.21.15.17:0/2891102482 shutdown_connections 2024-03-16T10:11:42.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb83d9640 1 --2- 172.21.15.17:0/2891102482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb013e2f0 0x7ffbb014eb30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb83d9640 1 --2- 172.21.15.17:0/2891102482 >> v2:172.21.15.174:3300/0 conn(0x7ffbb0066fc0 0x7ffbb00af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb83d9640 1 -- 172.21.15.17:0/2891102482 >> 172.21.15.17:0/2891102482 conn(0x7ffbb00b5e10 msgr2=0x7ffbb00b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:42.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb83d9640 1 -- 172.21.15.17:0/2891102482 shutdown_connections 2024-03-16T10:11:42.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.204+0000 7ffbb83d9640 1 -- 172.21.15.17:0/2891102482 wait complete. 2024-03-16T10:11:42.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb83d9640 1 Processor -- start 2024-03-16T10:11:42.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb83d9640 1 -- start start 2024-03-16T10:11:42.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb83d9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbb0066fc0 0x7ffbb009e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb83d9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffbb00673a0 0x7ffbb01e06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb83d9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb013e2f0 0x7ffbb009cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb83d9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb0151890 con 0x7ffbb013e2f0 2024-03-16T10:11:42.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb83d9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb0151a10 con 0x7ffbb0066fc0 2024-03-16T10:11:42.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb83d9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb0151b90 con 0x7ffbb00673a0 2024-03-16T10:11:42.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb6976640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb013e2f0 0x7ffbb009cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb6976640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb013e2f0 0x7ffbb009cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52432/0 (socket says 172.21.15.17:52432) 2024-03-16T10:11:42.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb6976640 1 -- 172.21.15.17:0/1462941094 learned_addr learned my addr 172.21.15.17:0/1462941094 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:42.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb6175640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbb0066fc0 0x7ffbb009e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb5974640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffbb00673a0 0x7ffbb01e06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb6976640 1 -- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffbb00673a0 msgr2=0x7ffbb01e06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb6976640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffbb00673a0 0x7ffbb01e06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb6976640 1 -- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbb0066fc0 msgr2=0x7ffbb009e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb6976640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbb0066fc0 0x7ffbb009e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb6976640 1 -- 172.21.15.17:0/1462941094 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb009eb10 con 0x7ffbb013e2f0 2024-03-16T10:11:42.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb5974640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffbb00673a0 0x7ffbb01e06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:42.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb6175640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbb0066fc0 0x7ffbb009e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:42.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb6976640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb013e2f0 0x7ffbb009cf70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffba801b010 tx=0x7ffba8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:42.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffba6ffd640 1 -- 172.21.15.17:0/1462941094 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba80bc030 con 0x7ffbb013e2f0 2024-03-16T10:11:42.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffba6ffd640 1 -- 172.21.15.17:0/1462941094 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba80b8470 con 0x7ffbb013e2f0 2024-03-16T10:11:42.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb83d9640 1 -- 172.21.15.17:0/1462941094 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb01e11b0 con 0x7ffbb013e2f0 2024-03-16T10:11:42.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffba6ffd640 1 -- 172.21.15.17:0/1462941094 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba80b8820 con 0x7ffbb013e2f0 2024-03-16T10:11:42.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.208+0000 7ffbb83d9640 1 -- 172.21.15.17:0/1462941094 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb009d4b0 con 0x7ffbb013e2f0 2024-03-16T10:11:42.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.212+0000 7ffbb83d9640 1 -- 172.21.15.17:0/1462941094 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb006a130 con 0x7ffbb013e2f0 2024-03-16T10:11:42.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.212+0000 7ffba6ffd640 1 -- 172.21.15.17:0/1462941094 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7ffba80bf070 con 0x7ffbb013e2f0 2024-03-16T10:11:42.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.212+0000 7ffba6ffd640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ffb94067660 0x7ffb94069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.212+0000 7ffba6ffd640 1 -- 172.21.15.17:0/1462941094 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7ffba81207a0 con 0x7ffbb013e2f0 2024-03-16T10:11:42.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.212+0000 7ffbb6175640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ffb94067660 0x7ffb94069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.212+0000 7ffbb6175640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ffb94067660 0x7ffb94069b20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffba000b9f0 tx=0x7ffba0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:42.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.216+0000 7ffba6ffd640 1 -- 172.21.15.17:0/1462941094 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ffba80165e0 con 0x7ffbb013e2f0 2024-03-16T10:11:42.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.452+0000 7ffbb83d9640 1 -- 172.21.15.17:0/1462941094 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbb0142200 con 0x7ffbb013e2f0 2024-03-16T10:11:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.452+0000 7ffba6ffd640 1 -- 172.21.15.17:0/1462941094 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+3531 (secure 0 0 0) 0x7ffba80f15d0 con 0x7ffbb013e2f0 2024-03-16T10:11:42.459 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:42.459 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4738,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1674220578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1674220578},{"type":"v1","addr":"172.21.15.174:6835","nonce":1674220578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:11:39.392755+0000","modified":"2024-03-16T10:11:41.419952+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6833/578943207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":578943207},{"type":"v1","addr":"172.21.15.17:6833","nonce":578943207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4953,"qdb_cluster":[4953]},"id":2}]} 2024-03-16T10:11:42.459 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 15 2024-03-16T10:11:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.456+0000 7ffbb83d9640 1 -- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ffb94067660 msgr2=0x7ffb94069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.456+0000 7ffbb83d9640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ffb94067660 0x7ffb94069b20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffba000b9f0 tx=0x7ffba0008040 comp rx=0 tx=0).stop 2024-03-16T10:11:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.456+0000 7ffbb83d9640 1 -- 172.21.15.17:0/1462941094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb013e2f0 msgr2=0x7ffbb009cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.456+0000 7ffbb83d9640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb013e2f0 0x7ffbb009cf70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffba801b010 tx=0x7ffba8019700 comp rx=0 tx=0).stop 2024-03-16T10:11:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.456+0000 7ffbb83d9640 1 -- 172.21.15.17:0/1462941094 shutdown_connections 2024-03-16T10:11:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.456+0000 7ffbb83d9640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ffb94067660 0x7ffb94069b20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.456+0000 7ffbb83d9640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbb013e2f0 0x7ffbb009cf70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.456+0000 7ffbb83d9640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ffbb00673a0 0x7ffbb01e06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.456+0000 7ffbb83d9640 1 --2- 172.21.15.17:0/1462941094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffbb0066fc0 0x7ffbb009e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.456+0000 7ffbb83d9640 1 -- 172.21.15.17:0/1462941094 >> 172.21.15.17:0/1462941094 conn(0x7ffbb00b5e10 msgr2=0x7ffbb006cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:42.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.456+0000 7ffbb83d9640 1 -- 172.21.15.17:0/1462941094 shutdown_connections 2024-03-16T10:11:42.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.456+0000 7ffbb83d9640 1 -- 172.21.15.17:0/1462941094 wait complete. 2024-03-16T10:11:42.476 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:42.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd74938b640 1 Processor -- start 2024-03-16T10:11:42.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd74938b640 1 -- start start 2024-03-16T10:11:42.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd74938b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74406b290 0x7fd744152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd74938b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd74406e6f0 con 0x7fd74406e310 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd74938b640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd74406b8b0 con 0x7fd7440731f0 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd74938b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd7440b5d30 con 0x7fd74406b290 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd742ffd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd74406e310 0x7fd744147440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48454/0 (socket says 172.21.15.17:48454) 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd742ffd640 1 -- 172.21.15.17:0/60425792 learned_addr learned my addr 172.21.15.17:0/60425792 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd7437fe640 1 --2- 172.21.15.17:0/60425792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74406b290 0x7fd744152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 -- 172.21.15.17:0/60425792 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 541768638 0 0) 0x7fd74406e6f0 con 0x7fd74406e310 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 -- 172.21.15.17:0/60425792 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd72c003530 con 0x7fd74406e310 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 -- 172.21.15.17:0/60425792 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1393422994 0 0) 0x7fd72c003530 con 0x7fd74406e310 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 -- 172.21.15.17:0/60425792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74406b290 msgr2=0x7fd744152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 --2- 172.21.15.17:0/60425792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74406b290 0x7fd744152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 -- 172.21.15.17:0/60425792 >> v1:172.21.15.174:6789/0 conn(0x7fd7440731f0 legacy=0x7fd7440b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 -- 172.21.15.17:0/60425792 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74406d5d0 con 0x7fd74406e310 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd7437fe640 1 --2- 172.21.15.17:0/60425792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74406b290 0x7fd744152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:42.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 -- 172.21.15.17:0/60425792 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fd734001680 con 0x7fd74406e310 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 -- 172.21.15.17:0/60425792 >> v1:172.21.15.17:6789/0 conn(0x7fd74406e310 legacy=0x7fd744147440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 --2- 172.21.15.17:0/60425792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72c0039f0 0x7fd72c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 --2- 172.21.15.17:0/60425792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74406b290 0x7fd72c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 --2- 172.21.15.17:0/60425792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c004d50 0x7fd72c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 -- 172.21.15.17:0/60425792 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd7440b5d30 con 0x7fd72c004d50 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 -- 172.21.15.17:0/60425792 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c00bb60 con 0x7fd72c0039f0 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd741ffb640 1 -- 172.21.15.17:0/60425792 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd72c00bd60 con 0x7fd74406b290 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd7437fe640 1 --2- 172.21.15.17:0/60425792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c004d50 0x7fd72c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd742ffd640 1 --2- 172.21.15.17:0/60425792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72c0039f0 0x7fd72c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd7427fc640 1 --2- 172.21.15.17:0/60425792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74406b290 0x7fd72c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd7437fe640 1 -- 172.21.15.17:0/60425792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74406b290 msgr2=0x7fd72c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd7437fe640 1 --2- 172.21.15.17:0/60425792 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74406b290 0x7fd72c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd7437fe640 1 -- 172.21.15.17:0/60425792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72c0039f0 msgr2=0x7fd72c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd7437fe640 1 --2- 172.21.15.17:0/60425792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72c0039f0 0x7fd72c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.596+0000 7fd7437fe640 1 -- 172.21.15.17:0/60425792 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c003530 con 0x7fd72c004d50 2024-03-16T10:11:42.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd7437fe640 1 --2- 172.21.15.17:0/60425792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c004d50 0x7fd72c0051d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0098d0 tx=0x7fd73c01c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934b4cef1c24ecb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:42.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd741ffb640 1 -- 172.21.15.17:0/60425792 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c00f040 con 0x7fd72c004d50 2024-03-16T10:11:42.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd741ffb640 1 -- 172.21.15.17:0/60425792 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c0af070 con 0x7fd72c004d50 2024-03-16T10:11:42.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd741ffb640 1 -- 172.21.15.17:0/60425792 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c0aa870 con 0x7fd72c004d50 2024-03-16T10:11:42.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd74938b640 1 -- 172.21.15.17:0/60425792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c004d50 msgr2=0x7fd72c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd74938b640 1 --2- 172.21.15.17:0/60425792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c004d50 0x7fd72c0051d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0098d0 tx=0x7fd73c01c5d0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd7437fe640 1 -- 172.21.15.17:0/60425792 reap_dead start 2024-03-16T10:11:42.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd74938b640 1 -- 172.21.15.17:0/60425792 shutdown_connections 2024-03-16T10:11:42.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd74938b640 1 -- 172.21.15.17:0/60425792 >> 172.21.15.17:0/60425792 conn(0x7fd7440ba0e0 msgr2=0x7fd7440ba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:42.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd74938b640 1 -- 172.21.15.17:0/60425792 shutdown_connections 2024-03-16T10:11:42.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd74938b640 1 -- 172.21.15.17:0/60425792 wait complete. 2024-03-16T10:11:42.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd74938b640 1 Processor -- start 2024-03-16T10:11:42.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd74938b640 1 -- start start 2024-03-16T10:11:42.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd74938b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c004d50 0x7fd7441e02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd74938b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74413da50 0x7fd74413bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd74938b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd744148ac0 0x7fd744148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd74938b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd7440b6340 con 0x7fd72c004d50 2024-03-16T10:11:42.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd74938b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd7440b61c0 con 0x7fd744148ac0 2024-03-16T10:11:42.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.600+0000 7fd74938b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd744150ce0 con 0x7fd74413da50 2024-03-16T10:11:42.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd742ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c004d50 0x7fd7441e02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd742ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c004d50 0x7fd7441e02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52458/0 (socket says 172.21.15.17:52458) 2024-03-16T10:11:42.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd742ffd640 1 -- 172.21.15.17:0/1072492265 learned_addr learned my addr 172.21.15.17:0/1072492265 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:42.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd7437fe640 1 --2- 172.21.15.17:0/1072492265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd744148ac0 0x7fd744148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd7427fc640 1 --2- 172.21.15.17:0/1072492265 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74413da50 0x7fd74413bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd742ffd640 1 -- 172.21.15.17:0/1072492265 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74413da50 msgr2=0x7fd74413bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd742ffd640 1 --2- 172.21.15.17:0/1072492265 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74413da50 0x7fd74413bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd742ffd640 1 -- 172.21.15.17:0/1072492265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd744148ac0 msgr2=0x7fd744148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd742ffd640 1 --2- 172.21.15.17:0/1072492265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd744148ac0 0x7fd744148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd742ffd640 1 -- 172.21.15.17:0/1072492265 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7441e0db0 con 0x7fd72c004d50 2024-03-16T10:11:42.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd742ffd640 1 --2- 172.21.15.17:0/1072492265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c004d50 0x7fd7441e02d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd7340124c0 tx=0x7fd734012980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:42.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd72b7fe640 1 -- 172.21.15.17:0/1072492265 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73401c070 con 0x7fd72c004d50 2024-03-16T10:11:42.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd72b7fe640 1 -- 172.21.15.17:0/1072492265 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd734014810 con 0x7fd72c004d50 2024-03-16T10:11:42.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd74938b640 1 -- 172.21.15.17:0/1072492265 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74413e280 con 0x7fd72c004d50 2024-03-16T10:11:42.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd72b7fe640 1 -- 172.21.15.17:0/1072492265 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd734014c00 con 0x7fd72c004d50 2024-03-16T10:11:42.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd74938b640 1 -- 172.21.15.17:0/1072492265 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd74413ec20 con 0x7fd72c004d50 2024-03-16T10:11:42.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd74938b640 1 -- 172.21.15.17:0/1072492265 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd74406a130 con 0x7fd72c004d50 2024-03-16T10:11:42.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd72b7fe640 1 -- 172.21.15.17:0/1072492265 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd73401f070 con 0x7fd72c004d50 2024-03-16T10:11:42.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd72b7fe640 1 --2- 172.21.15.17:0/1072492265 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd730067690 0x7fd730069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd72b7fe640 1 -- 172.21.15.17:0/1072492265 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fd734088150 con 0x7fd72c004d50 2024-03-16T10:11:42.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.604+0000 7fd7427fc640 1 --2- 172.21.15.17:0/1072492265 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd730067690 0x7fd730069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.608+0000 7fd7427fc640 1 --2- 172.21.15.17:0/1072492265 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd730067690 0x7fd730069b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd738001f00 tx=0x7fd738009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:42.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.608+0000 7fd72b7fe640 1 -- 172.21.15.17:0/1072492265 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd7441e0db0 con 0x7fd72c004d50 2024-03-16T10:11:42.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.768+0000 7fd74938b640 1 -- 172.21.15.17:0/1072492265 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd744145f30 con 0x7fd72c004d50 2024-03-16T10:11:42.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.772+0000 7fd72b7fe640 1 -- 172.21.15.17:0/1072492265 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16669 (secure 0 0 0) 0x7fd734058810 con 0x7fd72c004d50 2024-03-16T10:11:42.777 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:42.777 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":30,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:11:41.019229+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:11:37.284106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T10:11:38.392554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:42.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.772+0000 7fd74938b640 1 -- 172.21.15.17:0/1072492265 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd730067690 msgr2=0x7fd730069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.772+0000 7fd74938b640 1 --2- 172.21.15.17:0/1072492265 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd730067690 0x7fd730069b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd738001f00 tx=0x7fd738009040 comp rx=0 tx=0).stop 2024-03-16T10:11:42.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.772+0000 7fd74938b640 1 -- 172.21.15.17:0/1072492265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c004d50 msgr2=0x7fd7441e02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.772+0000 7fd74938b640 1 --2- 172.21.15.17:0/1072492265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c004d50 0x7fd7441e02d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd7340124c0 tx=0x7fd734012980 comp rx=0 tx=0).stop 2024-03-16T10:11:42.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.772+0000 7fd74938b640 1 -- 172.21.15.17:0/1072492265 shutdown_connections 2024-03-16T10:11:42.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.772+0000 7fd74938b640 1 --2- 172.21.15.17:0/1072492265 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd730067690 0x7fd730069b50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.772+0000 7fd74938b640 1 --2- 172.21.15.17:0/1072492265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd744148ac0 0x7fd744148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.772+0000 7fd74938b640 1 --2- 172.21.15.17:0/1072492265 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd74413da50 0x7fd74413bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.772+0000 7fd74938b640 1 --2- 172.21.15.17:0/1072492265 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd72c004d50 0x7fd7441e02d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.772+0000 7fd74938b640 1 -- 172.21.15.17:0/1072492265 >> 172.21.15.17:0/1072492265 conn(0x7fd7440ba0e0 msgr2=0x7fd7440b5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:42.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.772+0000 7fd74938b640 1 -- 172.21.15.17:0/1072492265 shutdown_connections 2024-03-16T10:11:42.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.772+0000 7fd74938b640 1 -- 172.21.15.17:0/1072492265 wait complete. 2024-03-16T10:11:42.795 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-16T10:11:42.795 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:11:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.900+0000 7f5ba1ee1640 1 Processor -- start 2024-03-16T10:11:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5ba1ee1640 1 -- start start 2024-03-16T10:11:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5ba1ee1640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5b9c0b5970 0x7f5b9c0aff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5ba1ee1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9c147850 0x7f5b9c13e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5ba1ee1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b9c147ce0 con 0x7f5b9c0af030 2024-03-16T10:11:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5ba1ee1640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f5b9c06f520 con 0x7f5b9c0b5970 2024-03-16T10:11:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5ba1ee1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5b9c0b5d50 con 0x7f5b9c147850 2024-03-16T10:11:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5b9affd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f5b9c0af030 0x7f5b9c06ec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48468/0 (socket says 172.21.15.17:48468) 2024-03-16T10:11:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5b9affd640 1 -- 172.21.15.17:0/2326051805 learned_addr learned my addr 172.21.15.17:0/2326051805 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5b9bfff640 1 --2- 172.21.15.17:0/2326051805 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9c147850 0x7f5b9c13e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5b9b7fe640 1 --2- 172.21.15.17:0/2326051805 >> v2:172.21.15.174:3300/0 conn(0x7f5b9c0b5970 0x7f5b9c0aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5b9a7fc640 1 -- 172.21.15.17:0/2326051805 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 593190810 0 0) 0x7f5b9c147ce0 con 0x7f5b9c0af030 2024-03-16T10:11:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5b9a7fc640 1 -- 172.21.15.17:0/2326051805 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b88003530 con 0x7f5b9c0af030 2024-03-16T10:11:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5b9bfff640 1 -- 172.21.15.17:0/2326051805 >> v2:172.21.15.174:3300/0 conn(0x7f5b9c0b5970 msgr2=0x7f5b9c0aff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5b9a7fc640 1 -- 172.21.15.17:0/2326051805 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1433561555 0 0) 0x7f5b88003530 con 0x7f5b9c0af030 2024-03-16T10:11:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5b9bfff640 1 --2- 172.21.15.17:0/2326051805 >> v2:172.21.15.174:3300/0 conn(0x7f5b9c0b5970 0x7f5b9c0aff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5b9bfff640 1 -- 172.21.15.17:0/2326051805 >> v1:172.21.15.17:6789/0 conn(0x7f5b9c0af030 legacy=0x7f5b9c06ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.904+0000 7f5b9bfff640 1 -- 172.21.15.17:0/2326051805 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c0af750 con 0x7f5b9c147850 2024-03-16T10:11:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9bfff640 1 --2- 172.21.15.17:0/2326051805 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9c147850 0x7f5b9c13e120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009d90 tx=0x7f5b9401c770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3435e370ab2ca17 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9a7fc640 1 -- 172.21.15.17:0/2326051805 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b940ac070 con 0x7f5b9c147850 2024-03-16T10:11:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9a7fc640 1 -- 172.21.15.17:0/2326051805 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b940a6450 con 0x7f5b9c147850 2024-03-16T10:11:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9a7fc640 1 -- 172.21.15.17:0/2326051805 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84001680 con 0x7f5b9c147850 2024-03-16T10:11:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/2326051805 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9c147850 msgr2=0x7f5b9c13e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 --2- 172.21.15.17:0/2326051805 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9c147850 0x7f5b9c13e120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009d90 tx=0x7f5b9401c770 comp rx=0 tx=0).stop 2024-03-16T10:11:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/2326051805 shutdown_connections 2024-03-16T10:11:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 --2- 172.21.15.17:0/2326051805 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9c147850 0x7f5b9c13e120 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 --2- 172.21.15.17:0/2326051805 >> v2:172.21.15.174:3300/0 conn(0x7f5b9c0b5970 0x7f5b9c0aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/2326051805 >> 172.21.15.17:0/2326051805 conn(0x7f5b9c0b6800 msgr2=0x7f5b9c0b6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/2326051805 shutdown_connections 2024-03-16T10:11:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/2326051805 wait complete. 2024-03-16T10:11:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 Processor -- start 2024-03-16T10:11:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 -- start start 2024-03-16T10:11:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9c0af030 0x7f5b9c13edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b9c0b5970 0x7f5b9c1ddbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9c147850 0x7f5b9c1437c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c151730 con 0x7f5b9c0af030 2024-03-16T10:11:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c1515b0 con 0x7f5b9c0b5970 2024-03-16T10:11:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5b9c1518b0 con 0x7f5b9c147850 2024-03-16T10:11:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9c0af030 0x7f5b9c13edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9c0af030 0x7f5b9c13edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52468/0 (socket says 172.21.15.17:52468) 2024-03-16T10:11:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9b7fe640 1 -- 172.21.15.17:0/4253926929 learned_addr learned my addr 172.21.15.17:0/4253926929 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9affd640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b9c0b5970 0x7f5b9c1ddbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9bfff640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9c147850 0x7f5b9c1437c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9b7fe640 1 -- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9c147850 msgr2=0x7f5b9c1437c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9b7fe640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9c147850 0x7f5b9c1437c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9b7fe640 1 -- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b9c0b5970 msgr2=0x7f5b9c1ddbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9b7fe640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b9c0b5970 0x7f5b9c1ddbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9b7fe640 1 -- 172.21.15.17:0/4253926929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c146ad0 con 0x7f5b9c0af030 2024-03-16T10:11:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9bfff640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9c147850 0x7f5b9c1437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b9b7fe640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9c0af030 0x7f5b9c13edc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00e930 tx=0x7f5b8c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b93fff640 1 -- 172.21.15.17:0/4253926929 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c017070 con 0x7f5b9c0af030 2024-03-16T10:11:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b93fff640 1 -- 172.21.15.17:0/4253926929 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b8c00ca00 con 0x7f5b9c0af030 2024-03-16T10:11:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/4253926929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c144680 con 0x7f5b9c0af030 2024-03-16T10:11:42.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5b93fff640 1 -- 172.21.15.17:0/4253926929 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00cdb0 con 0x7f5b9c0af030 2024-03-16T10:11:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.908+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/4253926929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c13f700 con 0x7f5b9c0af030 2024-03-16T10:11:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.912+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/4253926929 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9c06a130 con 0x7f5b9c0af030 2024-03-16T10:11:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.912+0000 7f5b93fff640 1 -- 172.21.15.17:0/4253926929 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f5b8c01a070 con 0x7f5b9c0af030 2024-03-16T10:11:42.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.912+0000 7f5b93fff640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5b7c067660 0x7f5b7c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.912+0000 7f5b93fff640 1 -- 172.21.15.17:0/4253926929 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f5b8c085920 con 0x7f5b9c0af030 2024-03-16T10:11:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.912+0000 7f5b9affd640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5b7c067660 0x7f5b7c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.912+0000 7f5b9affd640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5b7c067660 0x7f5b7c069b20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c147390 tx=0x7f5b84009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:42.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:42.916+0000 7f5b93fff640 1 -- 172.21.15.17:0/4253926929 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5b8c056670 con 0x7f5b9c0af030 2024-03-16T10:11:43.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.072+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/4253926929 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b9c141cf0 con 0x7f5b7c067660 2024-03-16T10:11:43.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.092+0000 7f5b93fff640 1 -- 172.21.15.17:0/4253926929 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249410 (secure 0 0 0) 0x7f5b9c141cf0 con 0x7f5b7c067660 2024-03-16T10:11:43.098 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:43.100 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-16T10:11:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.092+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5b7c067660 msgr2=0x7f5b7c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.092+0000 7f5ba1ee1640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5b7c067660 0x7f5b7c069b20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c147390 tx=0x7f5b84009040 comp rx=0 tx=0).stop 2024-03-16T10:11:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.092+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/4253926929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9c0af030 msgr2=0x7f5b9c13edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.092+0000 7f5ba1ee1640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9c0af030 0x7f5b9c13edc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00e930 tx=0x7f5b8c00edf0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.092+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/4253926929 shutdown_connections 2024-03-16T10:11:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.092+0000 7f5ba1ee1640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5b7c067660 0x7f5b7c069b20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.092+0000 7f5ba1ee1640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5b9c147850 0x7f5b9c1437c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.092+0000 7f5ba1ee1640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b9c0b5970 0x7f5b9c1ddbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.092+0000 7f5ba1ee1640 1 --2- 172.21.15.17:0/4253926929 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5b9c0af030 0x7f5b9c13edc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.092+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/4253926929 >> 172.21.15.17:0/4253926929 conn(0x7f5b9c0b6800 msgr2=0x7f5b9c06d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.092+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/4253926929 shutdown_connections 2024-03-16T10:11:43.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.092+0000 7f5ba1ee1640 1 -- 172.21.15.17:0/4253926929 wait complete. 2024-03-16T10:11:43.116 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-03-16T10:11:42.029504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591804,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":201,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236416,"kb_used_data":3592,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738304,"statfs":{"total":773094113280,"available":772852023296,"internally_reserved":0,"allocated":3678208,"data_stored":2105600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1436,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017091"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.029409+0000","last_change":"2024-03-16T10:11:40.029409+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.029409+0000","last_undegraded":"2024-03-16T10:11:40.029409+0000","last_fullsized":"2024-03-16T10:11:40.029409+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046589+0000","last_change":"2024-03-16T10:11:40.042825+0000","last_active":"2024-03-16T10:11:40.046589+0000","last_peered":"2024-03-16T10:11:40.046589+0000","last_clean":"2024-03-16T10:11:40.046589+0000","last_became_active":"2024-03-16T10:11:40.042533+0000","last_became_peered":"2024-03-16T10:11:40.042533+0000","last_unstale":"2024-03-16T10:11:40.046589+0000","last_undegraded":"2024-03-16T10:11:40.046589+0000","last_fullsized":"2024-03-16T10:11:40.046589+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:09:41.244065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.044573+0000","last_change":"2024-03-16T10:11:40.044573+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.044573+0000","last_undegraded":"2024-03-16T10:11:40.044573+0000","last_fullsized":"2024-03-16T10:11:40.044573+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048335+0000","last_change":"2024-03-16T10:11:40.045382+0000","last_active":"2024-03-16T10:11:40.048335+0000","last_peered":"2024-03-16T10:11:40.048335+0000","last_clean":"2024-03-16T10:11:40.048335+0000","last_became_active":"2024-03-16T10:11:40.045183+0000","last_became_peered":"2024-03-16T10:11:40.045183+0000","last_unstale":"2024-03-16T10:11:40.048335+0000","last_undegraded":"2024-03-16T10:11:40.048335+0000","last_fullsized":"2024-03-16T10:11:40.048335+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:04.144054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045289+0000","last_change":"2024-03-16T10:11:40.044298+0000","last_active":"2024-03-16T10:11:40.045289+0000","last_peered":"2024-03-16T10:11:40.045289+0000","last_clean":"2024-03-16T10:11:40.045289+0000","last_became_active":"2024-03-16T10:11:40.043984+0000","last_became_peered":"2024-03-16T10:11:40.043984+0000","last_unstale":"2024-03-16T10:11:40.045289+0000","last_undegraded":"2024-03-16T10:11:40.045289+0000","last_fullsized":"2024-03-16T10:11:40.045289+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:50.672600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.028131+0000","last_change":"2024-03-16T10:11:40.028131+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.028131+0000","last_undegraded":"2024-03-16T10:11:40.028131+0000","last_fullsized":"2024-03-16T10:11:40.028131+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":["3","5"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.040072+0000","last_change":"2024-03-16T10:11:40.040072+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.040072+0000","last_undegraded":"2024-03-16T10:11:40.040072+0000","last_fullsized":"2024-03-16T10:11:40.040072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.045399+0000","last_change":"2024-03-16T10:11:40.038809+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.045399+0000","last_undegraded":"2024-03-16T10:11:40.045399+0000","last_fullsized":"2024-03-16T10:11:40.045399+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.047641+0000","last_change":"2024-03-16T10:11:40.047641+0000","last_active":"2024-03-16T10:11:40.047641+0000","last_peered":"2024-03-16T10:11:40.047641+0000","last_clean":"2024-03-16T10:11:40.047641+0000","last_became_active":"2024-03-16T10:11:40.046666+0000","last_became_peered":"2024-03-16T10:11:40.046666+0000","last_unstale":"2024-03-16T10:11:40.047641+0000","last_undegraded":"2024-03-16T10:11:40.047641+0000","last_fullsized":"2024-03-16T10:11:40.047641+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:02.965529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048493+0000","last_change":"2024-03-16T10:11:40.047611+0000","last_active":"2024-03-16T10:11:40.048493+0000","last_peered":"2024-03-16T10:11:40.048493+0000","last_clean":"2024-03-16T10:11:40.048493+0000","last_became_active":"2024-03-16T10:11:40.047371+0000","last_became_peered":"2024-03-16T10:11:40.047371+0000","last_unstale":"2024-03-16T10:11:40.048493+0000","last_undegraded":"2024-03-16T10:11:40.048493+0000","last_fullsized":"2024-03-16T10:11:40.048493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:34:24.572872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.049110+0000","last_change":"2024-03-16T10:11:40.049110+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.049110+0000","last_undegraded":"2024-03-16T10:11:40.049110+0000","last_fullsized":"2024-03-16T10:11:40.049110+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.043630+0000","last_change":"2024-03-16T10:11:40.043630+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.043630+0000","last_undegraded":"2024-03-16T10:11:40.043630+0000","last_fullsized":"2024-03-16T10:11:40.043630+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.039941+0000","last_change":"2024-03-16T10:11:40.038996+0000","last_active":"2024-03-16T10:11:40.039941+0000","last_peered":"2024-03-16T10:11:40.039941+0000","last_clean":"2024-03-16T10:11:40.039941+0000","last_became_active":"2024-03-16T10:11:40.036020+0000","last_became_peered":"2024-03-16T10:11:40.036020+0000","last_unstale":"2024-03-16T10:11:40.039941+0000","last_undegraded":"2024-03-16T10:11:40.039941+0000","last_fullsized":"2024-03-16T10:11:40.039941+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:28:23.989331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046436+0000","last_change":"2024-03-16T10:11:40.042699+0000","last_active":"2024-03-16T10:11:40.046436+0000","last_peered":"2024-03-16T10:11:40.046436+0000","last_clean":"2024-03-16T10:11:40.046436+0000","last_became_active":"2024-03-16T10:11:40.042299+0000","last_became_peered":"2024-03-16T10:11:40.042299+0000","last_unstale":"2024-03-16T10:11:40.046436+0000","last_undegraded":"2024-03-16T10:11:40.046436+0000","last_fullsized":"2024-03-16T10:11:40.046436+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:39:53.674994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.035844+0000","last_change":"2024-03-16T10:11:40.035844+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.035844+0000","last_undegraded":"2024-03-16T10:11:40.035844+0000","last_fullsized":"2024-03-16T10:11:40.035844+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.041855+0000","last_change":"2024-03-16T10:11:40.041855+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.041855+0000","last_undegraded":"2024-03-16T10:11:40.041855+0000","last_fullsized":"2024-03-16T10:11:40.041855+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.049662+0000","last_change":"2024-03-16T10:11:40.049662+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.049662+0000","last_undegraded":"2024-03-16T10:11:40.049662+0000","last_fullsized":"2024-03-16T10:11:40.049662+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049450+0000","last_change":"2024-03-16T10:11:40.046723+0000","last_active":"2024-03-16T10:11:40.049450+0000","last_peered":"2024-03-16T10:11:40.049450+0000","last_clean":"2024-03-16T10:11:40.049450+0000","last_became_active":"2024-03-16T10:11:40.043586+0000","last_became_peered":"2024-03-16T10:11:40.043586+0000","last_unstale":"2024-03-16T10:11:40.049450+0000","last_undegraded":"2024-03-16T10:11:40.049450+0000","last_fullsized":"2024-03-16T10:11:40.049450+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:06.536564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046506+0000","last_change":"2024-03-16T10:11:40.043055+0000","last_active":"2024-03-16T10:11:40.046506+0000","last_peered":"2024-03-16T10:11:40.046506+0000","last_clean":"2024-03-16T10:11:40.046506+0000","last_became_active":"2024-03-16T10:11:40.041082+0000","last_became_peered":"2024-03-16T10:11:40.041082+0000","last_unstale":"2024-03-16T10:11:40.046506+0000","last_undegraded":"2024-03-16T10:11:40.046506+0000","last_fullsized":"2024-03-16T10:11:40.046506+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:35:29.488404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.023769+0000","last_change":"2024-03-16T10:11:40.023769+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.023769+0000","last_undegraded":"2024-03-16T10:11:40.023769+0000","last_fullsized":"2024-03-16T10:11:40.023769+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046160+0000","last_change":"2024-03-16T10:11:40.046160+0000","last_active":"2024-03-16T10:11:40.046160+0000","last_peered":"2024-03-16T10:11:40.046160+0000","last_clean":"2024-03-16T10:11:40.046160+0000","last_became_active":"2024-03-16T10:11:40.045457+0000","last_became_peered":"2024-03-16T10:11:40.045457+0000","last_unstale":"2024-03-16T10:11:40.046160+0000","last_undegraded":"2024-03-16T10:11:40.046160+0000","last_fullsized":"2024-03-16T10:11:40.046160+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:58.221102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.050020+0000","last_change":"2024-03-16T10:11:40.050020+0000","last_active":"2024-03-16T10:11:40.050020+0000","last_peered":"2024-03-16T10:11:40.050020+0000","last_clean":"2024-03-16T10:11:40.050020+0000","last_became_active":"2024-03-16T10:11:40.047232+0000","last_became_peered":"2024-03-16T10:11:40.047232+0000","last_unstale":"2024-03-16T10:11:40.050020+0000","last_undegraded":"2024-03-16T10:11:40.050020+0000","last_fullsized":"2024-03-16T10:11:40.050020+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:49.926346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.027173+0000","last_change":"2024-03-16T10:11:40.027173+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.027173+0000","last_undegraded":"2024-03-16T10:11:40.027173+0000","last_fullsized":"2024-03-16T10:11:40.027173+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.050081+0000","last_change":"2024-03-16T10:11:40.041024+0000","last_active":"2024-03-16T10:11:40.050081+0000","last_peered":"2024-03-16T10:11:40.050081+0000","last_clean":"2024-03-16T10:11:40.050081+0000","last_became_active":"2024-03-16T10:11:40.040230+0000","last_became_peered":"2024-03-16T10:11:40.040230+0000","last_unstale":"2024-03-16T10:11:40.050081+0000","last_undegraded":"2024-03-16T10:11:40.050081+0000","last_fullsized":"2024-03-16T10:11:40.050081+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:19.819589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.040254+0000","last_change":"2024-03-16T10:11:40.036983+0000","last_active":"2024-03-16T10:11:40.040254+0000","last_peered":"2024-03-16T10:11:40.040254+0000","last_clean":"2024-03-16T10:11:40.040254+0000","last_became_active":"2024-03-16T10:11:40.036701+0000","last_became_peered":"2024-03-16T10:11:40.036701+0000","last_unstale":"2024-03-16T10:11:40.040254+0000","last_undegraded":"2024-03-16T10:11:40.040254+0000","last_fullsized":"2024-03-16T10:11:40.040254+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:54.938794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.028496+0000","last_change":"2024-03-16T10:11:40.028496+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.028496+0000","last_undegraded":"2024-03-16T10:11:40.028496+0000","last_fullsized":"2024-03-16T10:11:40.028496+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048445+0000","last_change":"2024-03-16T10:11:40.046735+0000","last_active":"2024-03-16T10:11:40.048445+0000","last_peered":"2024-03-16T10:11:40.048445+0000","last_clean":"2024-03-16T10:11:40.048445+0000","last_became_active":"2024-03-16T10:11:40.046202+0000","last_became_peered":"2024-03-16T10:11:40.046202+0000","last_unstale":"2024-03-16T10:11:40.048445+0000","last_undegraded":"2024-03-16T10:11:40.048445+0000","last_fullsized":"2024-03-16T10:11:40.048445+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:48.954207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.042755+0000","last_change":"2024-03-16T10:11:40.042755+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.042755+0000","last_undegraded":"2024-03-16T10:11:40.042755+0000","last_fullsized":"2024-03-16T10:11:40.042755+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.067258+0000","last_change":"2024-03-16T10:11:40.067258+0000","last_active":"2024-03-16T10:11:40.067258+0000","last_peered":"2024-03-16T10:11:40.067258+0000","last_clean":"2024-03-16T10:11:40.067258+0000","last_became_active":"2024-03-16T10:11:40.066423+0000","last_became_peered":"2024-03-16T10:11:40.066423+0000","last_unstale":"2024-03-16T10:11:40.067258+0000","last_undegraded":"2024-03-16T10:11:40.067258+0000","last_fullsized":"2024-03-16T10:11:40.067258+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:04.144054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.041848+0000","last_change":"2024-03-16T10:11:40.041848+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.041848+0000","last_undegraded":"2024-03-16T10:11:40.041848+0000","last_fullsized":"2024-03-16T10:11:40.041848+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.034714+0000","last_change":"2024-03-16T10:11:40.028766+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.034714+0000","last_undegraded":"2024-03-16T10:11:40.034714+0000","last_fullsized":"2024-03-16T10:11:40.034714+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":["6","2"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.039748+0000","last_change":"2024-03-16T10:11:40.039748+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.039748+0000","last_undegraded":"2024-03-16T10:11:40.039748+0000","last_fullsized":"2024-03-16T10:11:40.039748+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.027172+0000","last_change":"2024-03-16T10:11:40.027172+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.027172+0000","last_undegraded":"2024-03-16T10:11:40.027172+0000","last_fullsized":"2024-03-16T10:11:40.027172+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"29'4","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030235+0000","last_change":"2024-03-16T10:11:39.061998+0000","last_active":"2024-03-16T10:11:40.030235+0000","last_peered":"2024-03-16T10:11:40.030235+0000","last_clean":"2024-03-16T10:11:40.030235+0000","last_became_active":"2024-03-16T10:11:39.061249+0000","last_became_peered":"2024-03-16T10:11:39.061249+0000","last_unstale":"2024-03-16T10:11:40.030235+0000","last_undegraded":"2024-03-16T10:11:40.030235+0000","last_fullsized":"2024-03-16T10:11:40.030235+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.061676+0000","last_change":"2024-03-16T10:11:40.061676+0000","last_active":"2024-03-16T10:11:40.061676+0000","last_peered":"2024-03-16T10:11:40.061676+0000","last_clean":"2024-03-16T10:11:40.061676+0000","last_became_active":"2024-03-16T10:11:40.060865+0000","last_became_peered":"2024-03-16T10:11:40.060865+0000","last_unstale":"2024-03-16T10:11:40.061676+0000","last_undegraded":"2024-03-16T10:11:40.061676+0000","last_fullsized":"2024-03-16T10:11:40.061676+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:40.792776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027530+0000","last_change":"2024-03-16T10:11:39.048117+0000","last_active":"2024-03-16T10:11:40.027530+0000","last_peered":"2024-03-16T10:11:40.027530+0000","last_clean":"2024-03-16T10:11:40.027530+0000","last_became_active":"2024-03-16T10:11:39.047798+0000","last_became_peered":"2024-03-16T10:11:39.047798+0000","last_unstale":"2024-03-16T10:11:40.027530+0000","last_undegraded":"2024-03-16T10:11:40.027530+0000","last_fullsized":"2024-03-16T10:11:40.027530+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.062870+0000","last_change":"2024-03-16T10:11:40.062870+0000","last_active":"2024-03-16T10:11:40.062870+0000","last_peered":"2024-03-16T10:11:40.062870+0000","last_clean":"2024-03-16T10:11:40.062870+0000","last_became_active":"2024-03-16T10:11:40.054629+0000","last_became_peered":"2024-03-16T10:11:40.054629+0000","last_unstale":"2024-03-16T10:11:40.062870+0000","last_undegraded":"2024-03-16T10:11:40.062870+0000","last_fullsized":"2024-03-16T10:11:40.062870+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:40.792776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"29'2","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049288+0000","last_change":"2024-03-16T10:11:39.038358+0000","last_active":"2024-03-16T10:11:40.049288+0000","last_peered":"2024-03-16T10:11:40.049288+0000","last_clean":"2024-03-16T10:11:40.049288+0000","last_became_active":"2024-03-16T10:11:39.038055+0000","last_became_peered":"2024-03-16T10:11:39.038055+0000","last_unstale":"2024-03-16T10:11:40.049288+0000","last_undegraded":"2024-03-16T10:11:40.049288+0000","last_fullsized":"2024-03-16T10:11:40.049288+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:11:55.739859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048424+0000","last_change":"2024-03-16T10:11:40.046475+0000","last_active":"2024-03-16T10:11:40.048424+0000","last_peered":"2024-03-16T10:11:40.048424+0000","last_clean":"2024-03-16T10:11:40.048424+0000","last_became_active":"2024-03-16T10:11:40.045876+0000","last_became_peered":"2024-03-16T10:11:40.045876+0000","last_unstale":"2024-03-16T10:11:40.048424+0000","last_undegraded":"2024-03-16T10:11:40.048424+0000","last_fullsized":"2024-03-16T10:11:40.048424+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:35:43.212538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"29'2","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045706+0000","last_change":"2024-03-16T10:11:39.030352+0000","last_active":"2024-03-16T10:11:40.045706+0000","last_peered":"2024-03-16T10:11:40.045706+0000","last_clean":"2024-03-16T10:11:40.045706+0000","last_became_active":"2024-03-16T10:11:39.029031+0000","last_became_peered":"2024-03-16T10:11:39.029031+0000","last_unstale":"2024-03-16T10:11:40.045706+0000","last_undegraded":"2024-03-16T10:11:40.045706+0000","last_fullsized":"2024-03-16T10:11:40.045706+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:07:13.816782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.041163+0000","last_change":"2024-03-16T10:11:40.041163+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.041163+0000","last_undegraded":"2024-03-16T10:11:40.041163+0000","last_fullsized":"2024-03-16T10:11:40.041163+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032683+0000","last_change":"2024-03-16T10:11:39.061267+0000","last_active":"2024-03-16T10:11:40.032683+0000","last_peered":"2024-03-16T10:11:40.032683+0000","last_clean":"2024-03-16T10:11:40.032683+0000","last_became_active":"2024-03-16T10:11:39.060323+0000","last_became_peered":"2024-03-16T10:11:39.060323+0000","last_unstale":"2024-03-16T10:11:40.032683+0000","last_undegraded":"2024-03-16T10:11:40.032683+0000","last_fullsized":"2024-03-16T10:11:40.032683+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:33:59.939254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.040415+0000","last_change":"2024-03-16T10:11:40.040415+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.040415+0000","last_undegraded":"2024-03-16T10:11:40.040415+0000","last_fullsized":"2024-03-16T10:11:40.040415+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027685+0000","last_change":"2024-03-16T10:11:39.061259+0000","last_active":"2024-03-16T10:11:40.027685+0000","last_peered":"2024-03-16T10:11:40.027685+0000","last_clean":"2024-03-16T10:11:40.027685+0000","last_became_active":"2024-03-16T10:11:39.060484+0000","last_became_peered":"2024-03-16T10:11:39.060484+0000","last_unstale":"2024-03-16T10:11:40.027685+0000","last_undegraded":"2024-03-16T10:11:40.027685+0000","last_fullsized":"2024-03-16T10:11:40.027685+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:54.254670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":6,"reported_epoch":29,"state":"active","last_fresh":"2024-03-16T10:11:40.070794+0000","last_change":"2024-03-16T10:11:40.070642+0000","last_active":"2024-03-16T10:11:40.070794+0000","last_peered":"2024-03-16T10:11:40.070794+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"2024-03-16T10:11:40.070642+0000","last_became_peered":"2024-03-16T10:11:40.070642+0000","last_unstale":"2024-03-16T10:11:40.070794+0000","last_undegraded":"2024-03-16T10:11:40.070794+0000","last_fullsized":"2024-03-16T10:11:40.070794+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.034336+0000","last_change":"2024-03-16T10:11:39.033585+0000","last_active":"2024-03-16T10:11:40.034336+0000","last_peered":"2024-03-16T10:11:40.034336+0000","last_clean":"2024-03-16T10:11:40.034336+0000","last_became_active":"2024-03-16T10:11:39.033131+0000","last_became_peered":"2024-03-16T10:11:39.033131+0000","last_unstale":"2024-03-16T10:11:40.034336+0000","last_undegraded":"2024-03-16T10:11:40.034336+0000","last_fullsized":"2024-03-16T10:11:40.034336+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:35:42.208648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.029382+0000","last_change":"2024-03-16T10:11:40.029382+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.029382+0000","last_undegraded":"2024-03-16T10:11:40.029382+0000","last_fullsized":"2024-03-16T10:11:40.029382+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.026886+0000","last_change":"2024-03-16T10:11:39.053374+0000","last_active":"2024-03-16T10:11:40.026886+0000","last_peered":"2024-03-16T10:11:40.026886+0000","last_clean":"2024-03-16T10:11:40.026886+0000","last_became_active":"2024-03-16T10:11:39.051683+0000","last_became_peered":"2024-03-16T10:11:39.051683+0000","last_unstale":"2024-03-16T10:11:40.026886+0000","last_undegraded":"2024-03-16T10:11:40.026886+0000","last_fullsized":"2024-03-16T10:11:40.026886+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:49.668710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.046622+0000","last_change":"2024-03-16T10:11:40.046622+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.046622+0000","last_undegraded":"2024-03-16T10:11:40.046622+0000","last_fullsized":"2024-03-16T10:11:40.046622+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.035769+0000","last_change":"2024-03-16T10:11:39.050941+0000","last_active":"2024-03-16T10:11:40.035769+0000","last_peered":"2024-03-16T10:11:40.035769+0000","last_clean":"2024-03-16T10:11:40.035769+0000","last_became_active":"2024-03-16T10:11:39.050659+0000","last_became_peered":"2024-03-16T10:11:39.050659+0000","last_unstale":"2024-03-16T10:11:40.035769+0000","last_undegraded":"2024-03-16T10:11:40.035769+0000","last_fullsized":"2024-03-16T10:11:40.035769+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:08.229218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046403+0000","last_change":"2024-03-16T10:11:40.043035+0000","last_active":"2024-03-16T10:11:40.046403+0000","last_peered":"2024-03-16T10:11:40.046403+0000","last_clean":"2024-03-16T10:11:40.046403+0000","last_became_active":"2024-03-16T10:11:40.040585+0000","last_became_peered":"2024-03-16T10:11:40.040585+0000","last_unstale":"2024-03-16T10:11:40.046403+0000","last_undegraded":"2024-03-16T10:11:40.046403+0000","last_fullsized":"2024-03-16T10:11:40.046403+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:36:55.327282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027611+0000","last_change":"2024-03-16T10:11:39.045927+0000","last_active":"2024-03-16T10:11:40.027611+0000","last_peered":"2024-03-16T10:11:40.027611+0000","last_clean":"2024-03-16T10:11:40.027611+0000","last_became_active":"2024-03-16T10:11:39.045502+0000","last_became_peered":"2024-03-16T10:11:39.045502+0000","last_unstale":"2024-03-16T10:11:40.027611+0000","last_undegraded":"2024-03-16T10:11:40.027611+0000","last_fullsized":"2024-03-16T10:11:40.027611+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049439+0000","last_change":"2024-03-16T10:11:40.041235+0000","last_active":"2024-03-16T10:11:40.049439+0000","last_peered":"2024-03-16T10:11:40.049439+0000","last_clean":"2024-03-16T10:11:40.049439+0000","last_became_active":"2024-03-16T10:11:40.040856+0000","last_became_peered":"2024-03-16T10:11:40.040856+0000","last_unstale":"2024-03-16T10:11:40.049439+0000","last_undegraded":"2024-03-16T10:11:40.049439+0000","last_fullsized":"2024-03-16T10:11:40.049439+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:38:40.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"29'3","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046332+0000","last_change":"2024-03-16T10:11:39.052193+0000","last_active":"2024-03-16T10:11:40.046332+0000","last_peered":"2024-03-16T10:11:40.046332+0000","last_clean":"2024-03-16T10:11:40.046332+0000","last_became_active":"2024-03-16T10:11:39.050633+0000","last_became_peered":"2024-03-16T10:11:39.050633+0000","last_unstale":"2024-03-16T10:11:40.046332+0000","last_undegraded":"2024-03-16T10:11:40.046332+0000","last_fullsized":"2024-03-16T10:11:40.046332+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:08.229218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.045474+0000","last_change":"2024-03-16T10:11:40.045474+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.045474+0000","last_undegraded":"2024-03-16T10:11:40.045474+0000","last_fullsized":"2024-03-16T10:11:40.045474+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"29'5","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.053253+0000","last_change":"2024-03-16T10:11:39.042282+0000","last_active":"2024-03-16T10:11:40.053253+0000","last_peered":"2024-03-16T10:11:40.053253+0000","last_clean":"2024-03-16T10:11:40.053253+0000","last_became_active":"2024-03-16T10:11:39.041924+0000","last_became_peered":"2024-03-16T10:11:39.041924+0000","last_unstale":"2024-03-16T10:11:40.053253+0000","last_undegraded":"2024-03-16T10:11:40.053253+0000","last_fullsized":"2024-03-16T10:11:40.053253+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:18.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.029679+0000","last_change":"2024-03-16T10:11:40.029679+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.029679+0000","last_undegraded":"2024-03-16T10:11:40.029679+0000","last_fullsized":"2024-03-16T10:11:40.029679+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.069420+0000","last_change":"2024-03-16T10:11:39.042217+0000","last_active":"2024-03-16T10:11:40.069420+0000","last_peered":"2024-03-16T10:11:40.069420+0000","last_clean":"2024-03-16T10:11:40.069420+0000","last_became_active":"2024-03-16T10:11:39.041514+0000","last_became_peered":"2024-03-16T10:11:39.041514+0000","last_unstale":"2024-03-16T10:11:40.069420+0000","last_undegraded":"2024-03-16T10:11:40.069420+0000","last_fullsized":"2024-03-16T10:11:40.069420+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:12:12.889055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049639+0000","last_change":"2024-03-16T10:11:40.048743+0000","last_active":"2024-03-16T10:11:40.049639+0000","last_peered":"2024-03-16T10:11:40.049639+0000","last_clean":"2024-03-16T10:11:40.049639+0000","last_became_active":"2024-03-16T10:11:40.047008+0000","last_became_peered":"2024-03-16T10:11:40.047008+0000","last_unstale":"2024-03-16T10:11:40.049639+0000","last_undegraded":"2024-03-16T10:11:40.049639+0000","last_fullsized":"2024-03-16T10:11:40.049639+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:56:04.173993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032632+0000","last_change":"2024-03-16T10:11:39.047491+0000","last_active":"2024-03-16T10:11:40.032632+0000","last_peered":"2024-03-16T10:11:40.032632+0000","last_clean":"2024-03-16T10:11:40.032632+0000","last_became_active":"2024-03-16T10:11:39.047085+0000","last_became_peered":"2024-03-16T10:11:39.047085+0000","last_unstale":"2024-03-16T10:11:40.032632+0000","last_undegraded":"2024-03-16T10:11:40.032632+0000","last_fullsized":"2024-03-16T10:11:40.032632+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.040610+0000","last_change":"2024-03-16T10:11:40.040610+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.040610+0000","last_undegraded":"2024-03-16T10:11:40.040610+0000","last_fullsized":"2024-03-16T10:11:40.040610+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.034687+0000","last_change":"2024-03-16T10:11:39.053564+0000","last_active":"2024-03-16T10:11:40.034687+0000","last_peered":"2024-03-16T10:11:40.034687+0000","last_clean":"2024-03-16T10:11:40.034687+0000","last_became_active":"2024-03-16T10:11:39.053183+0000","last_became_peered":"2024-03-16T10:11:39.053183+0000","last_unstale":"2024-03-16T10:11:40.034687+0000","last_undegraded":"2024-03-16T10:11:40.034687+0000","last_fullsized":"2024-03-16T10:11:40.034687+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:45:20.816477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":6,"reported_epoch":29,"state":"active","last_fresh":"2024-03-16T10:11:40.058574+0000","last_change":"2024-03-16T10:11:40.058392+0000","last_active":"2024-03-16T10:11:40.058574+0000","last_peered":"2024-03-16T10:11:40.058574+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"2024-03-16T10:11:40.058392+0000","last_became_peered":"2024-03-16T10:11:40.058392+0000","last_unstale":"2024-03-16T10:11:40.058574+0000","last_undegraded":"2024-03-16T10:11:40.058574+0000","last_fullsized":"2024-03-16T10:11:40.058574+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"29'8","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.053703+0000","last_change":"2024-03-16T10:11:39.035694+0000","last_active":"2024-03-16T10:11:40.053703+0000","last_peered":"2024-03-16T10:11:40.053703+0000","last_clean":"2024-03-16T10:11:40.053703+0000","last_became_active":"2024-03-16T10:11:39.035382+0000","last_became_peered":"2024-03-16T10:11:39.035382+0000","last_unstale":"2024-03-16T10:11:40.053703+0000","last_undegraded":"2024-03-16T10:11:40.053703+0000","last_fullsized":"2024-03-16T10:11:40.053703+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:47.950316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.064246+0000","last_change":"2024-03-16T10:11:40.064246+0000","last_active":"2024-03-16T10:11:40.064246+0000","last_peered":"2024-03-16T10:11:40.064246+0000","last_clean":"2024-03-16T10:11:40.064246+0000","last_became_active":"2024-03-16T10:11:40.063529+0000","last_became_peered":"2024-03-16T10:11:40.063529+0000","last_unstale":"2024-03-16T10:11:40.064246+0000","last_undegraded":"2024-03-16T10:11:40.064246+0000","last_fullsized":"2024-03-16T10:11:40.064246+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:54.938794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"29'1","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045024+0000","last_change":"2024-03-16T10:11:39.046131+0000","last_active":"2024-03-16T10:11:40.045024+0000","last_peered":"2024-03-16T10:11:40.045024+0000","last_clean":"2024-03-16T10:11:40.045024+0000","last_became_active":"2024-03-16T10:11:39.045785+0000","last_became_peered":"2024-03-16T10:11:39.045785+0000","last_unstale":"2024-03-16T10:11:40.045024+0000","last_undegraded":"2024-03-16T10:11:40.045024+0000","last_fullsized":"2024-03-16T10:11:40.045024+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:48.922456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.037470+0000","last_change":"2024-03-16T10:11:40.037470+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.037470+0000","last_undegraded":"2024-03-16T10:11:40.037470+0000","last_fullsized":"2024-03-16T10:11:40.037470+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"29'2","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030277+0000","last_change":"2024-03-16T10:11:39.051711+0000","last_active":"2024-03-16T10:11:40.030277+0000","last_peered":"2024-03-16T10:11:40.030277+0000","last_clean":"2024-03-16T10:11:40.030277+0000","last_became_active":"2024-03-16T10:11:39.050887+0000","last_became_peered":"2024-03-16T10:11:39.050887+0000","last_unstale":"2024-03-16T10:11:40.030277+0000","last_undegraded":"2024-03-16T10:11:40.030277+0000","last_fullsized":"2024-03-16T10:11:40.030277+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:03:20.074611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.031547+0000","last_change":"2024-03-16T10:11:40.031547+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.031547+0000","last_undegraded":"2024-03-16T10:11:40.031547+0000","last_fullsized":"2024-03-16T10:11:40.031547+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.026650+0000","last_change":"2024-03-16T10:11:39.059183+0000","last_active":"2024-03-16T10:11:40.026650+0000","last_peered":"2024-03-16T10:11:40.026650+0000","last_clean":"2024-03-16T10:11:40.026650+0000","last_became_active":"2024-03-16T10:11:39.052311+0000","last_became_peered":"2024-03-16T10:11:39.052311+0000","last_unstale":"2024-03-16T10:11:40.026650+0000","last_undegraded":"2024-03-16T10:11:40.026650+0000","last_fullsized":"2024-03-16T10:11:40.026650+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:28:22.985440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.024225+0000","last_change":"2024-03-16T10:11:40.024225+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.024225+0000","last_undegraded":"2024-03-16T10:11:40.024225+0000","last_fullsized":"2024-03-16T10:11:40.024225+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030295+0000","last_change":"2024-03-16T10:11:39.044835+0000","last_active":"2024-03-16T10:11:40.030295+0000","last_peered":"2024-03-16T10:11:40.030295+0000","last_clean":"2024-03-16T10:11:40.030295+0000","last_became_active":"2024-03-16T10:11:39.043928+0000","last_became_peered":"2024-03-16T10:11:39.043928+0000","last_unstale":"2024-03-16T10:11:40.030295+0000","last_undegraded":"2024-03-16T10:11:40.030295+0000","last_fullsized":"2024-03-16T10:11:40.030295+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:53.886085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.035827+0000","last_change":"2024-03-16T10:11:40.034188+0000","last_active":"2024-03-16T10:11:40.035827+0000","last_peered":"2024-03-16T10:11:40.035827+0000","last_clean":"2024-03-16T10:11:40.035827+0000","last_became_active":"2024-03-16T10:11:40.033873+0000","last_became_peered":"2024-03-16T10:11:40.033873+0000","last_unstale":"2024-03-16T10:11:40.035827+0000","last_undegraded":"2024-03-16T10:11:40.035827+0000","last_fullsized":"2024-03-16T10:11:40.035827+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:32:13.692871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"29'2","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.041749+0000","last_change":"2024-03-16T10:11:39.042466+0000","last_active":"2024-03-16T10:11:40.041749+0000","last_peered":"2024-03-16T10:11:40.041749+0000","last_clean":"2024-03-16T10:11:40.041749+0000","last_became_active":"2024-03-16T10:11:39.042100+0000","last_became_peered":"2024-03-16T10:11:39.042100+0000","last_unstale":"2024-03-16T10:11:40.041749+0000","last_undegraded":"2024-03-16T10:11:40.041749+0000","last_fullsized":"2024-03-16T10:11:40.041749+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:39.788886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032944+0000","last_change":"2024-03-16T10:11:40.031195+0000","last_active":"2024-03-16T10:11:40.032944+0000","last_peered":"2024-03-16T10:11:40.032944+0000","last_clean":"2024-03-16T10:11:40.032944+0000","last_became_active":"2024-03-16T10:11:40.030881+0000","last_became_peered":"2024-03-16T10:11:40.030881+0000","last_unstale":"2024-03-16T10:11:40.032944+0000","last_undegraded":"2024-03-16T10:11:40.032944+0000","last_fullsized":"2024-03-16T10:11:40.032944+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:46:51.399892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032725+0000","last_change":"2024-03-16T10:11:39.044909+0000","last_active":"2024-03-16T10:11:40.032725+0000","last_peered":"2024-03-16T10:11:40.032725+0000","last_clean":"2024-03-16T10:11:40.032725+0000","last_became_active":"2024-03-16T10:11:39.044151+0000","last_became_peered":"2024-03-16T10:11:39.044151+0000","last_unstale":"2024-03-16T10:11:40.032725+0000","last_undegraded":"2024-03-16T10:11:40.032725+0000","last_fullsized":"2024-03-16T10:11:40.032725+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032985+0000","last_change":"2024-03-16T10:11:40.031271+0000","last_active":"2024-03-16T10:11:40.032985+0000","last_peered":"2024-03-16T10:11:40.032985+0000","last_clean":"2024-03-16T10:11:40.032985+0000","last_became_active":"2024-03-16T10:11:40.030860+0000","last_became_peered":"2024-03-16T10:11:40.030860+0000","last_unstale":"2024-03-16T10:11:40.032985+0000","last_undegraded":"2024-03-16T10:11:40.032985+0000","last_fullsized":"2024-03-16T10:11:40.032985+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:50.216565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030652+0000","last_change":"2024-03-16T10:11:39.059806+0000","last_active":"2024-03-16T10:11:40.030652+0000","last_peered":"2024-03-16T10:11:40.030652+0000","last_clean":"2024-03-16T10:11:40.030652+0000","last_became_active":"2024-03-16T10:11:39.053858+0000","last_became_peered":"2024-03-16T10:11:39.053858+0000","last_unstale":"2024-03-16T10:11:40.030652+0000","last_undegraded":"2024-03-16T10:11:40.030652+0000","last_fullsized":"2024-03-16T10:11:40.030652+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:15:51.282542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.039125+0000","last_change":"2024-03-16T10:11:40.039125+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.039125+0000","last_undegraded":"2024-03-16T10:11:40.039125+0000","last_fullsized":"2024-03-16T10:11:40.039125+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.036628+0000","last_change":"2024-03-16T10:11:39.051315+0000","last_active":"2024-03-16T10:11:40.036628+0000","last_peered":"2024-03-16T10:11:40.036628+0000","last_clean":"2024-03-16T10:11:40.036628+0000","last_became_active":"2024-03-16T10:11:39.050550+0000","last_became_peered":"2024-03-16T10:11:39.050550+0000","last_unstale":"2024-03-16T10:11:40.036628+0000","last_undegraded":"2024-03-16T10:11:40.036628+0000","last_fullsized":"2024-03-16T10:11:40.036628+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:11:55.739859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.024301+0000","last_change":"2024-03-16T10:11:40.024301+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.024301+0000","last_undegraded":"2024-03-16T10:11:40.024301+0000","last_fullsized":"2024-03-16T10:11:40.024301+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"29'2","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032987+0000","last_change":"2024-03-16T10:11:39.060338+0000","last_active":"2024-03-16T10:11:40.032987+0000","last_peered":"2024-03-16T10:11:40.032987+0000","last_clean":"2024-03-16T10:11:40.032987+0000","last_became_active":"2024-03-16T10:11:39.060040+0000","last_became_peered":"2024-03-16T10:11:39.060040+0000","last_unstale":"2024-03-16T10:11:40.032987+0000","last_undegraded":"2024-03-16T10:11:40.032987+0000","last_fullsized":"2024-03-16T10:11:40.032987+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:39.788886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.036465+0000","last_change":"2024-03-16T10:11:40.036465+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.036465+0000","last_undegraded":"2024-03-16T10:11:40.036465+0000","last_fullsized":"2024-03-16T10:11:40.036465+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027701+0000","last_change":"2024-03-16T10:11:39.042670+0000","last_active":"2024-03-16T10:11:40.027701+0000","last_peered":"2024-03-16T10:11:40.027701+0000","last_clean":"2024-03-16T10:11:40.027701+0000","last_became_active":"2024-03-16T10:11:39.042250+0000","last_became_peered":"2024-03-16T10:11:39.042250+0000","last_unstale":"2024-03-16T10:11:40.027701+0000","last_undegraded":"2024-03-16T10:11:40.027701+0000","last_fullsized":"2024-03-16T10:11:40.027701+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.036767+0000","last_change":"2024-03-16T10:11:40.035336+0000","last_active":"2024-03-16T10:11:40.036767+0000","last_peered":"2024-03-16T10:11:40.036767+0000","last_clean":"2024-03-16T10:11:40.036767+0000","last_became_active":"2024-03-16T10:11:40.035053+0000","last_became_peered":"2024-03-16T10:11:40.035053+0000","last_unstale":"2024-03-16T10:11:40.036767+0000","last_undegraded":"2024-03-16T10:11:40.036767+0000","last_fullsized":"2024-03-16T10:11:40.036767+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:54.938794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032840+0000","last_change":"2024-03-16T10:11:39.034786+0000","last_active":"2024-03-16T10:11:40.032840+0000","last_peered":"2024-03-16T10:11:40.032840+0000","last_clean":"2024-03-16T10:11:40.032840+0000","last_became_active":"2024-03-16T10:11:39.034431+0000","last_became_peered":"2024-03-16T10:11:39.034431+0000","last_unstale":"2024-03-16T10:11:40.032840+0000","last_undegraded":"2024-03-16T10:11:40.032840+0000","last_fullsized":"2024-03-16T10:11:40.032840+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.037552+0000","last_change":"2024-03-16T10:11:40.031811+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.037552+0000","last_undegraded":"2024-03-16T10:11:40.037552+0000","last_fullsized":"2024-03-16T10:11:40.037552+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.028056+0000","last_change":"2024-03-16T10:11:39.054972+0000","last_active":"2024-03-16T10:11:40.028056+0000","last_peered":"2024-03-16T10:11:40.028056+0000","last_clean":"2024-03-16T10:11:40.028056+0000","last_became_active":"2024-03-16T10:11:39.054554+0000","last_became_peered":"2024-03-16T10:11:39.054554+0000","last_unstale":"2024-03-16T10:11:40.028056+0000","last_undegraded":"2024-03-16T10:11:40.028056+0000","last_fullsized":"2024-03-16T10:11:40.028056+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:48.922456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.027157+0000","last_change":"2024-03-16T10:11:40.027157+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.027157+0000","last_undegraded":"2024-03-16T10:11:40.027157+0000","last_fullsized":"2024-03-16T10:11:40.027157+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032789+0000","last_change":"2024-03-16T10:11:39.055099+0000","last_active":"2024-03-16T10:11:40.032789+0000","last_peered":"2024-03-16T10:11:40.032789+0000","last_clean":"2024-03-16T10:11:40.032789+0000","last_became_active":"2024-03-16T10:11:39.054743+0000","last_became_peered":"2024-03-16T10:11:39.054743+0000","last_unstale":"2024-03-16T10:11:40.032789+0000","last_undegraded":"2024-03-16T10:11:40.032789+0000","last_fullsized":"2024-03-16T10:11:40.032789+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049077+0000","last_change":"2024-03-16T10:11:40.045840+0000","last_active":"2024-03-16T10:11:40.049077+0000","last_peered":"2024-03-16T10:11:40.049077+0000","last_clean":"2024-03-16T10:11:40.049077+0000","last_became_active":"2024-03-16T10:11:40.040556+0000","last_became_peered":"2024-03-16T10:11:40.040556+0000","last_unstale":"2024-03-16T10:11:40.049077+0000","last_undegraded":"2024-03-16T10:11:40.049077+0000","last_fullsized":"2024-03-16T10:11:40.049077+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:44:53.636655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.031865+0000","last_change":"2024-03-16T10:11:39.053883+0000","last_active":"2024-03-16T10:11:40.031865+0000","last_peered":"2024-03-16T10:11:40.031865+0000","last_clean":"2024-03-16T10:11:40.031865+0000","last_became_active":"2024-03-16T10:11:39.052424+0000","last_became_peered":"2024-03-16T10:11:39.052424+0000","last_unstale":"2024-03-16T10:11:40.031865+0000","last_undegraded":"2024-03-16T10:11:40.031865+0000","last_fullsized":"2024-03-16T10:11:40.031865+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:35:59.698889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049061+0000","last_change":"2024-03-16T10:11:40.049061+0000","last_active":"2024-03-16T10:11:40.049061+0000","last_peered":"2024-03-16T10:11:40.049061+0000","last_clean":"2024-03-16T10:11:40.049061+0000","last_became_active":"2024-03-16T10:11:40.048340+0000","last_became_peered":"2024-03-16T10:11:40.048340+0000","last_unstale":"2024-03-16T10:11:40.049061+0000","last_undegraded":"2024-03-16T10:11:40.049061+0000","last_fullsized":"2024-03-16T10:11:40.049061+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:21.380536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.034511+0000","last_change":"2024-03-16T10:11:39.049474+0000","last_active":"2024-03-16T10:11:40.034511+0000","last_peered":"2024-03-16T10:11:40.034511+0000","last_clean":"2024-03-16T10:11:40.034511+0000","last_became_active":"2024-03-16T10:11:39.049180+0000","last_became_peered":"2024-03-16T10:11:39.049180+0000","last_unstale":"2024-03-16T10:11:40.034511+0000","last_undegraded":"2024-03-16T10:11:40.034511+0000","last_fullsized":"2024-03-16T10:11:40.034511+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:34:23.568982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.045609+0000","last_change":"2024-03-16T10:11:40.039231+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.045609+0000","last_undegraded":"2024-03-16T10:11:40.045609+0000","last_fullsized":"2024-03-16T10:11:40.045609+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027544+0000","last_change":"2024-03-16T10:11:39.055688+0000","last_active":"2024-03-16T10:11:40.027544+0000","last_peered":"2024-03-16T10:11:40.027544+0000","last_clean":"2024-03-16T10:11:40.027544+0000","last_became_active":"2024-03-16T10:11:39.054833+0000","last_became_peered":"2024-03-16T10:11:39.054833+0000","last_unstale":"2024-03-16T10:11:40.027544+0000","last_undegraded":"2024-03-16T10:11:40.027544+0000","last_fullsized":"2024-03-16T10:11:40.027544+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":185,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045097+0000","last_change":"2024-03-16T10:10:48.248686+0000","last_active":"2024-03-16T10:11:40.045097+0000","last_peered":"2024-03-16T10:11:40.045097+0000","last_clean":"2024-03-16T10:11:40.045097+0000","last_became_active":"2024-03-16T10:10:48.243054+0000","last_became_peered":"2024-03-16T10:10:48.243054+0000","last_unstale":"2024-03-16T10:11:40.045097+0000","last_undegraded":"2024-03-16T10:11:40.045097+0000","last_fullsized":"2024-03-16T10:11:40.045097+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_clean_scrub_stamp":"2024-03-16T10:10:46.517938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:42.906473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1436,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32064,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339776,"statfs":{"total":96636764160,"available":96603930624,"internally_reserved":0,"allocated":905216,"data_stored":705976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:11:43.119 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:11:43.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.280+0000 7f44c9f95640 1 Processor -- start 2024-03-16T10:11:43.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.280+0000 7f44c9f95640 1 -- start start 2024-03-16T10:11:43.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c9f95640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f44c40b72e0 0x7f44c4142360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c9f95640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44c406f5a0 con 0x7f44c4148a30 2024-03-16T10:11:43.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c9f95640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44c406fdc0 con 0x7f44c406e660 2024-03-16T10:11:43.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c9f95640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f44c40a0e20 con 0x7f44c40b72e0 2024-03-16T10:11:43.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c37fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f44c4148a30 0x7f44c4148e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48480/0 (socket says 172.21.15.17:48480) 2024-03-16T10:11:43.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c37fe640 1 -- 172.21.15.17:0/3026922774 learned_addr learned my addr 172.21.15.17:0/3026922774 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:43.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c2ffd640 1 --2- 172.21.15.17:0/3026922774 >> v2:172.21.15.174:3300/0 conn(0x7f44c40b72e0 0x7f44c4142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 -- 172.21.15.17:0/3026922774 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1762712390 0 0) 0x7f44c406f5a0 con 0x7f44c4148a30 2024-03-16T10:11:43.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 -- 172.21.15.17:0/3026922774 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44ac003530 con 0x7f44c4148a30 2024-03-16T10:11:43.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 -- 172.21.15.17:0/3026922774 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1060743207 0 0) 0x7f44ac003530 con 0x7f44c4148a30 2024-03-16T10:11:43.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 -- 172.21.15.17:0/3026922774 >> v2:172.21.15.174:3300/0 conn(0x7f44c40b72e0 msgr2=0x7f44c4142360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:43.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 --2- 172.21.15.17:0/3026922774 >> v2:172.21.15.174:3300/0 conn(0x7f44c40b72e0 0x7f44c4142360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 -- 172.21.15.17:0/3026922774 >> v1:172.21.15.174:6789/0 conn(0x7f44c406e660 legacy=0x7f44c414fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:43.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 -- 172.21.15.17:0/3026922774 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c4149520 con 0x7f44c4148a30 2024-03-16T10:11:43.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 -- 172.21.15.17:0/3026922774 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f44bc001680 con 0x7f44c4148a30 2024-03-16T10:11:43.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 -- 172.21.15.17:0/3026922774 >> v1:172.21.15.17:6789/0 conn(0x7f44c4148a30 legacy=0x7f44c4148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:43.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 --2- 172.21.15.17:0/3026922774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac003a10 0x7f44ac003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 --2- 172.21.15.17:0/3026922774 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44ac004810 0x7f44ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 --2- 172.21.15.17:0/3026922774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ac0052f0 0x7f44ac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 -- 172.21.15.17:0/3026922774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f44c40a0e20 con 0x7f44ac0052f0 2024-03-16T10:11:43.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 -- 172.21.15.17:0/3026922774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44ac00c1f0 con 0x7f44ac003a10 2024-03-16T10:11:43.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c27fc640 1 -- 172.21.15.17:0/3026922774 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f44ac00c3f0 con 0x7f44ac004810 2024-03-16T10:11:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c3fff640 1 --2- 172.21.15.17:0/3026922774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ac0052f0 0x7f44ac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c2ffd640 1 --2- 172.21.15.17:0/3026922774 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44ac004810 0x7f44ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c37fe640 1 --2- 172.21.15.17:0/3026922774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac003a10 0x7f44ac003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c3fff640 1 -- 172.21.15.17:0/3026922774 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44ac004810 msgr2=0x7f44ac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c3fff640 1 --2- 172.21.15.17:0/3026922774 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44ac004810 0x7f44ac004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c3fff640 1 -- 172.21.15.17:0/3026922774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac003a10 msgr2=0x7f44ac003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c3fff640 1 --2- 172.21.15.17:0/3026922774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac003a10 0x7f44ac003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c3fff640 1 -- 172.21.15.17:0/3026922774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ac003530 con 0x7f44ac0052f0 2024-03-16T10:11:43.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.284+0000 7f44c37fe640 1 --2- 172.21.15.17:0/3026922774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44ac003a10 0x7f44ac003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:43.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c3fff640 1 -- 172.21.15.17:0/3026922774 reap_dead start 2024-03-16T10:11:43.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c3fff640 1 --2- 172.21.15.17:0/3026922774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ac0052f0 0x7f44ac00b9d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f44b4001e40 tx=0x7f44b401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513499d8e3efdd9f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:43.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c27fc640 1 -- 172.21.15.17:0/3026922774 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4001a90 con 0x7f44ac0052f0 2024-03-16T10:11:43.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c27fc640 1 -- 172.21.15.17:0/3026922774 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44b40a7070 con 0x7f44ac0052f0 2024-03-16T10:11:43.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c27fc640 1 -- 172.21.15.17:0/3026922774 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4003dc0 con 0x7f44ac0052f0 2024-03-16T10:11:43.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 -- 172.21.15.17:0/3026922774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ac0052f0 msgr2=0x7f44ac00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:43.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 --2- 172.21.15.17:0/3026922774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ac0052f0 0x7f44ac00b9d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f44b4001e40 tx=0x7f44b401c760 comp rx=0 tx=0).stop 2024-03-16T10:11:43.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 -- 172.21.15.17:0/3026922774 shutdown_connections 2024-03-16T10:11:43.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 --2- 172.21.15.17:0/3026922774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44ac0052f0 0x7f44ac00b9d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 -- 172.21.15.17:0/3026922774 >> 172.21.15.17:0/3026922774 conn(0x7f44c40718a0 msgr2=0x7f44c4071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:43.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 -- 172.21.15.17:0/3026922774 shutdown_connections 2024-03-16T10:11:43.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 -- 172.21.15.17:0/3026922774 wait complete. 2024-03-16T10:11:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 Processor -- start 2024-03-16T10:11:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 -- start start 2024-03-16T10:11:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44ac0052f0 0x7f44c4138660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c41463c0 0x7f44c4146820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44c41e0b50 0x7f44c41e0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4142570 con 0x7f44c41463c0 2024-03-16T10:11:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44c40a0fa0 con 0x7f44c41e0b50 2024-03-16T10:11:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c9f95640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f44c41426f0 con 0x7f44ac0052f0 2024-03-16T10:11:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c41463c0 0x7f44c4146820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c2ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c41463c0 0x7f44c4146820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52486/0 (socket says 172.21.15.17:52486) 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c2ffd640 1 -- 172.21.15.17:0/3475737705 learned_addr learned my addr 172.21.15.17:0/3475737705 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c3fff640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44c41e0b50 0x7f44c41e0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c37fe640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44ac0052f0 0x7f44c4138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c2ffd640 1 -- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44ac0052f0 msgr2=0x7f44c4138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c2ffd640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44ac0052f0 0x7f44c4138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c2ffd640 1 -- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44c41e0b50 msgr2=0x7f44c41e0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c2ffd640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44c41e0b50 0x7f44c41e0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c2ffd640 1 -- 172.21.15.17:0/3475737705 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c4145bd0 con 0x7f44c41463c0 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c37fe640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44ac0052f0 0x7f44c4138660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c3fff640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44c41e0b50 0x7f44c41e0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.288+0000 7f44c2ffd640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c41463c0 0x7f44c4146820 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f44b8011570 tx=0x7f44b8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.292+0000 7f44abfff640 1 -- 172.21.15.17:0/3475737705 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8015930 con 0x7f44c41463c0 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.292+0000 7f44abfff640 1 -- 172.21.15.17:0/3475737705 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44b8015ad0 con 0x7f44c41463c0 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.292+0000 7f44c9f95640 1 -- 172.21.15.17:0/3475737705 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c4145eb0 con 0x7f44c41463c0 2024-03-16T10:11:43.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.292+0000 7f44abfff640 1 -- 172.21.15.17:0/3475737705 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b801d400 con 0x7f44c41463c0 2024-03-16T10:11:43.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.292+0000 7f44c9f95640 1 -- 172.21.15.17:0/3475737705 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c41458f0 con 0x7f44c41463c0 2024-03-16T10:11:43.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.292+0000 7f44c9f95640 1 -- 172.21.15.17:0/3475737705 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c406a130 con 0x7f44c41463c0 2024-03-16T10:11:43.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.292+0000 7f44abfff640 1 -- 172.21.15.17:0/3475737705 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f44b801d5a0 con 0x7f44c41463c0 2024-03-16T10:11:43.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.292+0000 7f44abfff640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f44b0067660 0x7f44b0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.292+0000 7f44abfff640 1 -- 172.21.15.17:0/3475737705 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f44b80861c0 con 0x7f44c41463c0 2024-03-16T10:11:43.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.292+0000 7f44c37fe640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f44b0067660 0x7f44b0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.292+0000 7f44c37fe640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f44b0067660 0x7f44b0069b20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f44bc000c10 tx=0x7f44bc0015c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:43.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.296+0000 7f44abfff640 1 -- 172.21.15.17:0/3475737705 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f44b8019030 con 0x7f44c41463c0 2024-03-16T10:11:43.333 INFO:tasks.ceph.mds.a.smithi017.stderr:2024-03-16T10:11:43.328+0000 7f3ac45d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-16T10:11:43.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.596+0000 7f44c9f95640 1 -- 172.21.15.17:0/3475737705 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44c4146d60 con 0x7f44b0067660 2024-03-16T10:11:43.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.612+0000 7f44abfff640 1 -- 172.21.15.17:0/3475737705 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249410 (secure 0 0 0) 0x7f44bc00f990 con 0x7f44b0067660 2024-03-16T10:11:43.620 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:43.623 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-16T10:11:43.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.620+0000 7f44c9f95640 1 -- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f44b0067660 msgr2=0x7f44b0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:43.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.620+0000 7f44c9f95640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f44b0067660 0x7f44b0069b20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f44bc000c10 tx=0x7f44bc0015c0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.620+0000 7f44c9f95640 1 -- 172.21.15.17:0/3475737705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c41463c0 msgr2=0x7f44c4146820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.620+0000 7f44c9f95640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c41463c0 0x7f44c4146820 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f44b8011570 tx=0x7f44b8011a30 comp rx=0 tx=0).stop 2024-03-16T10:11:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.620+0000 7f44c9f95640 1 -- 172.21.15.17:0/3475737705 shutdown_connections 2024-03-16T10:11:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.620+0000 7f44c9f95640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f44b0067660 0x7f44b0069b20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.620+0000 7f44c9f95640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44c41e0b50 0x7f44c41e0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.620+0000 7f44c9f95640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c41463c0 0x7f44c4146820 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.620+0000 7f44c9f95640 1 --2- 172.21.15.17:0/3475737705 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f44ac0052f0 0x7f44c4138660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.620+0000 7f44c9f95640 1 -- 172.21.15.17:0/3475737705 >> 172.21.15.17:0/3475737705 conn(0x7f44c40718a0 msgr2=0x7f44c40b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.620+0000 7f44c9f95640 1 -- 172.21.15.17:0/3475737705 shutdown_connections 2024-03-16T10:11:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.620+0000 7f44c9f95640 1 -- 172.21.15.17:0/3475737705 wait complete. 2024-03-16T10:11:43.644 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-03-16T10:11:42.029504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591804,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":201,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236416,"kb_used_data":3592,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738304,"statfs":{"total":773094113280,"available":772852023296,"internally_reserved":0,"allocated":3678208,"data_stored":2105600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1436,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017091"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.029409+0000","last_change":"2024-03-16T10:11:40.029409+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.029409+0000","last_undegraded":"2024-03-16T10:11:40.029409+0000","last_fullsized":"2024-03-16T10:11:40.029409+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046589+0000","last_change":"2024-03-16T10:11:40.042825+0000","last_active":"2024-03-16T10:11:40.046589+0000","last_peered":"2024-03-16T10:11:40.046589+0000","last_clean":"2024-03-16T10:11:40.046589+0000","last_became_active":"2024-03-16T10:11:40.042533+0000","last_became_peered":"2024-03-16T10:11:40.042533+0000","last_unstale":"2024-03-16T10:11:40.046589+0000","last_undegraded":"2024-03-16T10:11:40.046589+0000","last_fullsized":"2024-03-16T10:11:40.046589+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:09:41.244065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.044573+0000","last_change":"2024-03-16T10:11:40.044573+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.044573+0000","last_undegraded":"2024-03-16T10:11:40.044573+0000","last_fullsized":"2024-03-16T10:11:40.044573+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048335+0000","last_change":"2024-03-16T10:11:40.045382+0000","last_active":"2024-03-16T10:11:40.048335+0000","last_peered":"2024-03-16T10:11:40.048335+0000","last_clean":"2024-03-16T10:11:40.048335+0000","last_became_active":"2024-03-16T10:11:40.045183+0000","last_became_peered":"2024-03-16T10:11:40.045183+0000","last_unstale":"2024-03-16T10:11:40.048335+0000","last_undegraded":"2024-03-16T10:11:40.048335+0000","last_fullsized":"2024-03-16T10:11:40.048335+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:04.144054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045289+0000","last_change":"2024-03-16T10:11:40.044298+0000","last_active":"2024-03-16T10:11:40.045289+0000","last_peered":"2024-03-16T10:11:40.045289+0000","last_clean":"2024-03-16T10:11:40.045289+0000","last_became_active":"2024-03-16T10:11:40.043984+0000","last_became_peered":"2024-03-16T10:11:40.043984+0000","last_unstale":"2024-03-16T10:11:40.045289+0000","last_undegraded":"2024-03-16T10:11:40.045289+0000","last_fullsized":"2024-03-16T10:11:40.045289+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:50.672600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.028131+0000","last_change":"2024-03-16T10:11:40.028131+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.028131+0000","last_undegraded":"2024-03-16T10:11:40.028131+0000","last_fullsized":"2024-03-16T10:11:40.028131+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":["3","5"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.040072+0000","last_change":"2024-03-16T10:11:40.040072+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.040072+0000","last_undegraded":"2024-03-16T10:11:40.040072+0000","last_fullsized":"2024-03-16T10:11:40.040072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.045399+0000","last_change":"2024-03-16T10:11:40.038809+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.045399+0000","last_undegraded":"2024-03-16T10:11:40.045399+0000","last_fullsized":"2024-03-16T10:11:40.045399+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.047641+0000","last_change":"2024-03-16T10:11:40.047641+0000","last_active":"2024-03-16T10:11:40.047641+0000","last_peered":"2024-03-16T10:11:40.047641+0000","last_clean":"2024-03-16T10:11:40.047641+0000","last_became_active":"2024-03-16T10:11:40.046666+0000","last_became_peered":"2024-03-16T10:11:40.046666+0000","last_unstale":"2024-03-16T10:11:40.047641+0000","last_undegraded":"2024-03-16T10:11:40.047641+0000","last_fullsized":"2024-03-16T10:11:40.047641+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:02.965529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048493+0000","last_change":"2024-03-16T10:11:40.047611+0000","last_active":"2024-03-16T10:11:40.048493+0000","last_peered":"2024-03-16T10:11:40.048493+0000","last_clean":"2024-03-16T10:11:40.048493+0000","last_became_active":"2024-03-16T10:11:40.047371+0000","last_became_peered":"2024-03-16T10:11:40.047371+0000","last_unstale":"2024-03-16T10:11:40.048493+0000","last_undegraded":"2024-03-16T10:11:40.048493+0000","last_fullsized":"2024-03-16T10:11:40.048493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:34:24.572872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.049110+0000","last_change":"2024-03-16T10:11:40.049110+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.049110+0000","last_undegraded":"2024-03-16T10:11:40.049110+0000","last_fullsized":"2024-03-16T10:11:40.049110+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.043630+0000","last_change":"2024-03-16T10:11:40.043630+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.043630+0000","last_undegraded":"2024-03-16T10:11:40.043630+0000","last_fullsized":"2024-03-16T10:11:40.043630+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.039941+0000","last_change":"2024-03-16T10:11:40.038996+0000","last_active":"2024-03-16T10:11:40.039941+0000","last_peered":"2024-03-16T10:11:40.039941+0000","last_clean":"2024-03-16T10:11:40.039941+0000","last_became_active":"2024-03-16T10:11:40.036020+0000","last_became_peered":"2024-03-16T10:11:40.036020+0000","last_unstale":"2024-03-16T10:11:40.039941+0000","last_undegraded":"2024-03-16T10:11:40.039941+0000","last_fullsized":"2024-03-16T10:11:40.039941+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:28:23.989331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046436+0000","last_change":"2024-03-16T10:11:40.042699+0000","last_active":"2024-03-16T10:11:40.046436+0000","last_peered":"2024-03-16T10:11:40.046436+0000","last_clean":"2024-03-16T10:11:40.046436+0000","last_became_active":"2024-03-16T10:11:40.042299+0000","last_became_peered":"2024-03-16T10:11:40.042299+0000","last_unstale":"2024-03-16T10:11:40.046436+0000","last_undegraded":"2024-03-16T10:11:40.046436+0000","last_fullsized":"2024-03-16T10:11:40.046436+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:39:53.674994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.035844+0000","last_change":"2024-03-16T10:11:40.035844+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.035844+0000","last_undegraded":"2024-03-16T10:11:40.035844+0000","last_fullsized":"2024-03-16T10:11:40.035844+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.041855+0000","last_change":"2024-03-16T10:11:40.041855+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.041855+0000","last_undegraded":"2024-03-16T10:11:40.041855+0000","last_fullsized":"2024-03-16T10:11:40.041855+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.049662+0000","last_change":"2024-03-16T10:11:40.049662+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.049662+0000","last_undegraded":"2024-03-16T10:11:40.049662+0000","last_fullsized":"2024-03-16T10:11:40.049662+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049450+0000","last_change":"2024-03-16T10:11:40.046723+0000","last_active":"2024-03-16T10:11:40.049450+0000","last_peered":"2024-03-16T10:11:40.049450+0000","last_clean":"2024-03-16T10:11:40.049450+0000","last_became_active":"2024-03-16T10:11:40.043586+0000","last_became_peered":"2024-03-16T10:11:40.043586+0000","last_unstale":"2024-03-16T10:11:40.049450+0000","last_undegraded":"2024-03-16T10:11:40.049450+0000","last_fullsized":"2024-03-16T10:11:40.049450+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:06.536564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046506+0000","last_change":"2024-03-16T10:11:40.043055+0000","last_active":"2024-03-16T10:11:40.046506+0000","last_peered":"2024-03-16T10:11:40.046506+0000","last_clean":"2024-03-16T10:11:40.046506+0000","last_became_active":"2024-03-16T10:11:40.041082+0000","last_became_peered":"2024-03-16T10:11:40.041082+0000","last_unstale":"2024-03-16T10:11:40.046506+0000","last_undegraded":"2024-03-16T10:11:40.046506+0000","last_fullsized":"2024-03-16T10:11:40.046506+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:35:29.488404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.023769+0000","last_change":"2024-03-16T10:11:40.023769+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.023769+0000","last_undegraded":"2024-03-16T10:11:40.023769+0000","last_fullsized":"2024-03-16T10:11:40.023769+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046160+0000","last_change":"2024-03-16T10:11:40.046160+0000","last_active":"2024-03-16T10:11:40.046160+0000","last_peered":"2024-03-16T10:11:40.046160+0000","last_clean":"2024-03-16T10:11:40.046160+0000","last_became_active":"2024-03-16T10:11:40.045457+0000","last_became_peered":"2024-03-16T10:11:40.045457+0000","last_unstale":"2024-03-16T10:11:40.046160+0000","last_undegraded":"2024-03-16T10:11:40.046160+0000","last_fullsized":"2024-03-16T10:11:40.046160+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:58.221102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.050020+0000","last_change":"2024-03-16T10:11:40.050020+0000","last_active":"2024-03-16T10:11:40.050020+0000","last_peered":"2024-03-16T10:11:40.050020+0000","last_clean":"2024-03-16T10:11:40.050020+0000","last_became_active":"2024-03-16T10:11:40.047232+0000","last_became_peered":"2024-03-16T10:11:40.047232+0000","last_unstale":"2024-03-16T10:11:40.050020+0000","last_undegraded":"2024-03-16T10:11:40.050020+0000","last_fullsized":"2024-03-16T10:11:40.050020+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:49.926346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.027173+0000","last_change":"2024-03-16T10:11:40.027173+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.027173+0000","last_undegraded":"2024-03-16T10:11:40.027173+0000","last_fullsized":"2024-03-16T10:11:40.027173+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.050081+0000","last_change":"2024-03-16T10:11:40.041024+0000","last_active":"2024-03-16T10:11:40.050081+0000","last_peered":"2024-03-16T10:11:40.050081+0000","last_clean":"2024-03-16T10:11:40.050081+0000","last_became_active":"2024-03-16T10:11:40.040230+0000","last_became_peered":"2024-03-16T10:11:40.040230+0000","last_unstale":"2024-03-16T10:11:40.050081+0000","last_undegraded":"2024-03-16T10:11:40.050081+0000","last_fullsized":"2024-03-16T10:11:40.050081+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:19.819589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.040254+0000","last_change":"2024-03-16T10:11:40.036983+0000","last_active":"2024-03-16T10:11:40.040254+0000","last_peered":"2024-03-16T10:11:40.040254+0000","last_clean":"2024-03-16T10:11:40.040254+0000","last_became_active":"2024-03-16T10:11:40.036701+0000","last_became_peered":"2024-03-16T10:11:40.036701+0000","last_unstale":"2024-03-16T10:11:40.040254+0000","last_undegraded":"2024-03-16T10:11:40.040254+0000","last_fullsized":"2024-03-16T10:11:40.040254+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:54.938794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.028496+0000","last_change":"2024-03-16T10:11:40.028496+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.028496+0000","last_undegraded":"2024-03-16T10:11:40.028496+0000","last_fullsized":"2024-03-16T10:11:40.028496+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048445+0000","last_change":"2024-03-16T10:11:40.046735+0000","last_active":"2024-03-16T10:11:40.048445+0000","last_peered":"2024-03-16T10:11:40.048445+0000","last_clean":"2024-03-16T10:11:40.048445+0000","last_became_active":"2024-03-16T10:11:40.046202+0000","last_became_peered":"2024-03-16T10:11:40.046202+0000","last_unstale":"2024-03-16T10:11:40.048445+0000","last_undegraded":"2024-03-16T10:11:40.048445+0000","last_fullsized":"2024-03-16T10:11:40.048445+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:48.954207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.042755+0000","last_change":"2024-03-16T10:11:40.042755+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.042755+0000","last_undegraded":"2024-03-16T10:11:40.042755+0000","last_fullsized":"2024-03-16T10:11:40.042755+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.067258+0000","last_change":"2024-03-16T10:11:40.067258+0000","last_active":"2024-03-16T10:11:40.067258+0000","last_peered":"2024-03-16T10:11:40.067258+0000","last_clean":"2024-03-16T10:11:40.067258+0000","last_became_active":"2024-03-16T10:11:40.066423+0000","last_became_peered":"2024-03-16T10:11:40.066423+0000","last_unstale":"2024-03-16T10:11:40.067258+0000","last_undegraded":"2024-03-16T10:11:40.067258+0000","last_fullsized":"2024-03-16T10:11:40.067258+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:04.144054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.041848+0000","last_change":"2024-03-16T10:11:40.041848+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.041848+0000","last_undegraded":"2024-03-16T10:11:40.041848+0000","last_fullsized":"2024-03-16T10:11:40.041848+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.034714+0000","last_change":"2024-03-16T10:11:40.028766+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.034714+0000","last_undegraded":"2024-03-16T10:11:40.034714+0000","last_fullsized":"2024-03-16T10:11:40.034714+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":["6","2"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.039748+0000","last_change":"2024-03-16T10:11:40.039748+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.039748+0000","last_undegraded":"2024-03-16T10:11:40.039748+0000","last_fullsized":"2024-03-16T10:11:40.039748+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.027172+0000","last_change":"2024-03-16T10:11:40.027172+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.027172+0000","last_undegraded":"2024-03-16T10:11:40.027172+0000","last_fullsized":"2024-03-16T10:11:40.027172+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"29'4","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030235+0000","last_change":"2024-03-16T10:11:39.061998+0000","last_active":"2024-03-16T10:11:40.030235+0000","last_peered":"2024-03-16T10:11:40.030235+0000","last_clean":"2024-03-16T10:11:40.030235+0000","last_became_active":"2024-03-16T10:11:39.061249+0000","last_became_peered":"2024-03-16T10:11:39.061249+0000","last_unstale":"2024-03-16T10:11:40.030235+0000","last_undegraded":"2024-03-16T10:11:40.030235+0000","last_fullsized":"2024-03-16T10:11:40.030235+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.061676+0000","last_change":"2024-03-16T10:11:40.061676+0000","last_active":"2024-03-16T10:11:40.061676+0000","last_peered":"2024-03-16T10:11:40.061676+0000","last_clean":"2024-03-16T10:11:40.061676+0000","last_became_active":"2024-03-16T10:11:40.060865+0000","last_became_peered":"2024-03-16T10:11:40.060865+0000","last_unstale":"2024-03-16T10:11:40.061676+0000","last_undegraded":"2024-03-16T10:11:40.061676+0000","last_fullsized":"2024-03-16T10:11:40.061676+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:40.792776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027530+0000","last_change":"2024-03-16T10:11:39.048117+0000","last_active":"2024-03-16T10:11:40.027530+0000","last_peered":"2024-03-16T10:11:40.027530+0000","last_clean":"2024-03-16T10:11:40.027530+0000","last_became_active":"2024-03-16T10:11:39.047798+0000","last_became_peered":"2024-03-16T10:11:39.047798+0000","last_unstale":"2024-03-16T10:11:40.027530+0000","last_undegraded":"2024-03-16T10:11:40.027530+0000","last_fullsized":"2024-03-16T10:11:40.027530+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.062870+0000","last_change":"2024-03-16T10:11:40.062870+0000","last_active":"2024-03-16T10:11:40.062870+0000","last_peered":"2024-03-16T10:11:40.062870+0000","last_clean":"2024-03-16T10:11:40.062870+0000","last_became_active":"2024-03-16T10:11:40.054629+0000","last_became_peered":"2024-03-16T10:11:40.054629+0000","last_unstale":"2024-03-16T10:11:40.062870+0000","last_undegraded":"2024-03-16T10:11:40.062870+0000","last_fullsized":"2024-03-16T10:11:40.062870+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:40.792776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"29'2","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049288+0000","last_change":"2024-03-16T10:11:39.038358+0000","last_active":"2024-03-16T10:11:40.049288+0000","last_peered":"2024-03-16T10:11:40.049288+0000","last_clean":"2024-03-16T10:11:40.049288+0000","last_became_active":"2024-03-16T10:11:39.038055+0000","last_became_peered":"2024-03-16T10:11:39.038055+0000","last_unstale":"2024-03-16T10:11:40.049288+0000","last_undegraded":"2024-03-16T10:11:40.049288+0000","last_fullsized":"2024-03-16T10:11:40.049288+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:11:55.739859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048424+0000","last_change":"2024-03-16T10:11:40.046475+0000","last_active":"2024-03-16T10:11:40.048424+0000","last_peered":"2024-03-16T10:11:40.048424+0000","last_clean":"2024-03-16T10:11:40.048424+0000","last_became_active":"2024-03-16T10:11:40.045876+0000","last_became_peered":"2024-03-16T10:11:40.045876+0000","last_unstale":"2024-03-16T10:11:40.048424+0000","last_undegraded":"2024-03-16T10:11:40.048424+0000","last_fullsized":"2024-03-16T10:11:40.048424+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:35:43.212538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"29'2","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045706+0000","last_change":"2024-03-16T10:11:39.030352+0000","last_active":"2024-03-16T10:11:40.045706+0000","last_peered":"2024-03-16T10:11:40.045706+0000","last_clean":"2024-03-16T10:11:40.045706+0000","last_became_active":"2024-03-16T10:11:39.029031+0000","last_became_peered":"2024-03-16T10:11:39.029031+0000","last_unstale":"2024-03-16T10:11:40.045706+0000","last_undegraded":"2024-03-16T10:11:40.045706+0000","last_fullsized":"2024-03-16T10:11:40.045706+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:07:13.816782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.041163+0000","last_change":"2024-03-16T10:11:40.041163+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.041163+0000","last_undegraded":"2024-03-16T10:11:40.041163+0000","last_fullsized":"2024-03-16T10:11:40.041163+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032683+0000","last_change":"2024-03-16T10:11:39.061267+0000","last_active":"2024-03-16T10:11:40.032683+0000","last_peered":"2024-03-16T10:11:40.032683+0000","last_clean":"2024-03-16T10:11:40.032683+0000","last_became_active":"2024-03-16T10:11:39.060323+0000","last_became_peered":"2024-03-16T10:11:39.060323+0000","last_unstale":"2024-03-16T10:11:40.032683+0000","last_undegraded":"2024-03-16T10:11:40.032683+0000","last_fullsized":"2024-03-16T10:11:40.032683+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:33:59.939254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.040415+0000","last_change":"2024-03-16T10:11:40.040415+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.040415+0000","last_undegraded":"2024-03-16T10:11:40.040415+0000","last_fullsized":"2024-03-16T10:11:40.040415+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027685+0000","last_change":"2024-03-16T10:11:39.061259+0000","last_active":"2024-03-16T10:11:40.027685+0000","last_peered":"2024-03-16T10:11:40.027685+0000","last_clean":"2024-03-16T10:11:40.027685+0000","last_became_active":"2024-03-16T10:11:39.060484+0000","last_became_peered":"2024-03-16T10:11:39.060484+0000","last_unstale":"2024-03-16T10:11:40.027685+0000","last_undegraded":"2024-03-16T10:11:40.027685+0000","last_fullsized":"2024-03-16T10:11:40.027685+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:54.254670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":6,"reported_epoch":29,"state":"active","last_fresh":"2024-03-16T10:11:40.070794+0000","last_change":"2024-03-16T10:11:40.070642+0000","last_active":"2024-03-16T10:11:40.070794+0000","last_peered":"2024-03-16T10:11:40.070794+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"2024-03-16T10:11:40.070642+0000","last_became_peered":"2024-03-16T10:11:40.070642+0000","last_unstale":"2024-03-16T10:11:40.070794+0000","last_undegraded":"2024-03-16T10:11:40.070794+0000","last_fullsized":"2024-03-16T10:11:40.070794+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.034336+0000","last_change":"2024-03-16T10:11:39.033585+0000","last_active":"2024-03-16T10:11:40.034336+0000","last_peered":"2024-03-16T10:11:40.034336+0000","last_clean":"2024-03-16T10:11:40.034336+0000","last_became_active":"2024-03-16T10:11:39.033131+0000","last_became_peered":"2024-03-16T10:11:39.033131+0000","last_unstale":"2024-03-16T10:11:40.034336+0000","last_undegraded":"2024-03-16T10:11:40.034336+0000","last_fullsized":"2024-03-16T10:11:40.034336+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:35:42.208648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.029382+0000","last_change":"2024-03-16T10:11:40.029382+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.029382+0000","last_undegraded":"2024-03-16T10:11:40.029382+0000","last_fullsized":"2024-03-16T10:11:40.029382+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.026886+0000","last_change":"2024-03-16T10:11:39.053374+0000","last_active":"2024-03-16T10:11:40.026886+0000","last_peered":"2024-03-16T10:11:40.026886+0000","last_clean":"2024-03-16T10:11:40.026886+0000","last_became_active":"2024-03-16T10:11:39.051683+0000","last_became_peered":"2024-03-16T10:11:39.051683+0000","last_unstale":"2024-03-16T10:11:40.026886+0000","last_undegraded":"2024-03-16T10:11:40.026886+0000","last_fullsized":"2024-03-16T10:11:40.026886+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:49.668710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.046622+0000","last_change":"2024-03-16T10:11:40.046622+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.046622+0000","last_undegraded":"2024-03-16T10:11:40.046622+0000","last_fullsized":"2024-03-16T10:11:40.046622+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.035769+0000","last_change":"2024-03-16T10:11:39.050941+0000","last_active":"2024-03-16T10:11:40.035769+0000","last_peered":"2024-03-16T10:11:40.035769+0000","last_clean":"2024-03-16T10:11:40.035769+0000","last_became_active":"2024-03-16T10:11:39.050659+0000","last_became_peered":"2024-03-16T10:11:39.050659+0000","last_unstale":"2024-03-16T10:11:40.035769+0000","last_undegraded":"2024-03-16T10:11:40.035769+0000","last_fullsized":"2024-03-16T10:11:40.035769+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:08.229218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046403+0000","last_change":"2024-03-16T10:11:40.043035+0000","last_active":"2024-03-16T10:11:40.046403+0000","last_peered":"2024-03-16T10:11:40.046403+0000","last_clean":"2024-03-16T10:11:40.046403+0000","last_became_active":"2024-03-16T10:11:40.040585+0000","last_became_peered":"2024-03-16T10:11:40.040585+0000","last_unstale":"2024-03-16T10:11:40.046403+0000","last_undegraded":"2024-03-16T10:11:40.046403+0000","last_fullsized":"2024-03-16T10:11:40.046403+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:36:55.327282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027611+0000","last_change":"2024-03-16T10:11:39.045927+0000","last_active":"2024-03-16T10:11:40.027611+0000","last_peered":"2024-03-16T10:11:40.027611+0000","last_clean":"2024-03-16T10:11:40.027611+0000","last_became_active":"2024-03-16T10:11:39.045502+0000","last_became_peered":"2024-03-16T10:11:39.045502+0000","last_unstale":"2024-03-16T10:11:40.027611+0000","last_undegraded":"2024-03-16T10:11:40.027611+0000","last_fullsized":"2024-03-16T10:11:40.027611+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049439+0000","last_change":"2024-03-16T10:11:40.041235+0000","last_active":"2024-03-16T10:11:40.049439+0000","last_peered":"2024-03-16T10:11:40.049439+0000","last_clean":"2024-03-16T10:11:40.049439+0000","last_became_active":"2024-03-16T10:11:40.040856+0000","last_became_peered":"2024-03-16T10:11:40.040856+0000","last_unstale":"2024-03-16T10:11:40.049439+0000","last_undegraded":"2024-03-16T10:11:40.049439+0000","last_fullsized":"2024-03-16T10:11:40.049439+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:38:40.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"29'3","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046332+0000","last_change":"2024-03-16T10:11:39.052193+0000","last_active":"2024-03-16T10:11:40.046332+0000","last_peered":"2024-03-16T10:11:40.046332+0000","last_clean":"2024-03-16T10:11:40.046332+0000","last_became_active":"2024-03-16T10:11:39.050633+0000","last_became_peered":"2024-03-16T10:11:39.050633+0000","last_unstale":"2024-03-16T10:11:40.046332+0000","last_undegraded":"2024-03-16T10:11:40.046332+0000","last_fullsized":"2024-03-16T10:11:40.046332+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:08.229218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.045474+0000","last_change":"2024-03-16T10:11:40.045474+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.045474+0000","last_undegraded":"2024-03-16T10:11:40.045474+0000","last_fullsized":"2024-03-16T10:11:40.045474+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"29'5","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.053253+0000","last_change":"2024-03-16T10:11:39.042282+0000","last_active":"2024-03-16T10:11:40.053253+0000","last_peered":"2024-03-16T10:11:40.053253+0000","last_clean":"2024-03-16T10:11:40.053253+0000","last_became_active":"2024-03-16T10:11:39.041924+0000","last_became_peered":"2024-03-16T10:11:39.041924+0000","last_unstale":"2024-03-16T10:11:40.053253+0000","last_undegraded":"2024-03-16T10:11:40.053253+0000","last_fullsized":"2024-03-16T10:11:40.053253+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:18.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.029679+0000","last_change":"2024-03-16T10:11:40.029679+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.029679+0000","last_undegraded":"2024-03-16T10:11:40.029679+0000","last_fullsized":"2024-03-16T10:11:40.029679+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.069420+0000","last_change":"2024-03-16T10:11:39.042217+0000","last_active":"2024-03-16T10:11:40.069420+0000","last_peered":"2024-03-16T10:11:40.069420+0000","last_clean":"2024-03-16T10:11:40.069420+0000","last_became_active":"2024-03-16T10:11:39.041514+0000","last_became_peered":"2024-03-16T10:11:39.041514+0000","last_unstale":"2024-03-16T10:11:40.069420+0000","last_undegraded":"2024-03-16T10:11:40.069420+0000","last_fullsized":"2024-03-16T10:11:40.069420+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:12:12.889055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049639+0000","last_change":"2024-03-16T10:11:40.048743+0000","last_active":"2024-03-16T10:11:40.049639+0000","last_peered":"2024-03-16T10:11:40.049639+0000","last_clean":"2024-03-16T10:11:40.049639+0000","last_became_active":"2024-03-16T10:11:40.047008+0000","last_became_peered":"2024-03-16T10:11:40.047008+0000","last_unstale":"2024-03-16T10:11:40.049639+0000","last_undegraded":"2024-03-16T10:11:40.049639+0000","last_fullsized":"2024-03-16T10:11:40.049639+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:56:04.173993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032632+0000","last_change":"2024-03-16T10:11:39.047491+0000","last_active":"2024-03-16T10:11:40.032632+0000","last_peered":"2024-03-16T10:11:40.032632+0000","last_clean":"2024-03-16T10:11:40.032632+0000","last_became_active":"2024-03-16T10:11:39.047085+0000","last_became_peered":"2024-03-16T10:11:39.047085+0000","last_unstale":"2024-03-16T10:11:40.032632+0000","last_undegraded":"2024-03-16T10:11:40.032632+0000","last_fullsized":"2024-03-16T10:11:40.032632+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.040610+0000","last_change":"2024-03-16T10:11:40.040610+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.040610+0000","last_undegraded":"2024-03-16T10:11:40.040610+0000","last_fullsized":"2024-03-16T10:11:40.040610+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.034687+0000","last_change":"2024-03-16T10:11:39.053564+0000","last_active":"2024-03-16T10:11:40.034687+0000","last_peered":"2024-03-16T10:11:40.034687+0000","last_clean":"2024-03-16T10:11:40.034687+0000","last_became_active":"2024-03-16T10:11:39.053183+0000","last_became_peered":"2024-03-16T10:11:39.053183+0000","last_unstale":"2024-03-16T10:11:40.034687+0000","last_undegraded":"2024-03-16T10:11:40.034687+0000","last_fullsized":"2024-03-16T10:11:40.034687+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:45:20.816477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":6,"reported_epoch":29,"state":"active","last_fresh":"2024-03-16T10:11:40.058574+0000","last_change":"2024-03-16T10:11:40.058392+0000","last_active":"2024-03-16T10:11:40.058574+0000","last_peered":"2024-03-16T10:11:40.058574+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"2024-03-16T10:11:40.058392+0000","last_became_peered":"2024-03-16T10:11:40.058392+0000","last_unstale":"2024-03-16T10:11:40.058574+0000","last_undegraded":"2024-03-16T10:11:40.058574+0000","last_fullsized":"2024-03-16T10:11:40.058574+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"29'8","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.053703+0000","last_change":"2024-03-16T10:11:39.035694+0000","last_active":"2024-03-16T10:11:40.053703+0000","last_peered":"2024-03-16T10:11:40.053703+0000","last_clean":"2024-03-16T10:11:40.053703+0000","last_became_active":"2024-03-16T10:11:39.035382+0000","last_became_peered":"2024-03-16T10:11:39.035382+0000","last_unstale":"2024-03-16T10:11:40.053703+0000","last_undegraded":"2024-03-16T10:11:40.053703+0000","last_fullsized":"2024-03-16T10:11:40.053703+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:47.950316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.064246+0000","last_change":"2024-03-16T10:11:40.064246+0000","last_active":"2024-03-16T10:11:40.064246+0000","last_peered":"2024-03-16T10:11:40.064246+0000","last_clean":"2024-03-16T10:11:40.064246+0000","last_became_active":"2024-03-16T10:11:40.063529+0000","last_became_peered":"2024-03-16T10:11:40.063529+0000","last_unstale":"2024-03-16T10:11:40.064246+0000","last_undegraded":"2024-03-16T10:11:40.064246+0000","last_fullsized":"2024-03-16T10:11:40.064246+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:54.938794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"29'1","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045024+0000","last_change":"2024-03-16T10:11:39.046131+0000","last_active":"2024-03-16T10:11:40.045024+0000","last_peered":"2024-03-16T10:11:40.045024+0000","last_clean":"2024-03-16T10:11:40.045024+0000","last_became_active":"2024-03-16T10:11:39.045785+0000","last_became_peered":"2024-03-16T10:11:39.045785+0000","last_unstale":"2024-03-16T10:11:40.045024+0000","last_undegraded":"2024-03-16T10:11:40.045024+0000","last_fullsized":"2024-03-16T10:11:40.045024+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:48.922456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.037470+0000","last_change":"2024-03-16T10:11:40.037470+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.037470+0000","last_undegraded":"2024-03-16T10:11:40.037470+0000","last_fullsized":"2024-03-16T10:11:40.037470+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"29'2","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030277+0000","last_change":"2024-03-16T10:11:39.051711+0000","last_active":"2024-03-16T10:11:40.030277+0000","last_peered":"2024-03-16T10:11:40.030277+0000","last_clean":"2024-03-16T10:11:40.030277+0000","last_became_active":"2024-03-16T10:11:39.050887+0000","last_became_peered":"2024-03-16T10:11:39.050887+0000","last_unstale":"2024-03-16T10:11:40.030277+0000","last_undegraded":"2024-03-16T10:11:40.030277+0000","last_fullsized":"2024-03-16T10:11:40.030277+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:03:20.074611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.031547+0000","last_change":"2024-03-16T10:11:40.031547+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.031547+0000","last_undegraded":"2024-03-16T10:11:40.031547+0000","last_fullsized":"2024-03-16T10:11:40.031547+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.026650+0000","last_change":"2024-03-16T10:11:39.059183+0000","last_active":"2024-03-16T10:11:40.026650+0000","last_peered":"2024-03-16T10:11:40.026650+0000","last_clean":"2024-03-16T10:11:40.026650+0000","last_became_active":"2024-03-16T10:11:39.052311+0000","last_became_peered":"2024-03-16T10:11:39.052311+0000","last_unstale":"2024-03-16T10:11:40.026650+0000","last_undegraded":"2024-03-16T10:11:40.026650+0000","last_fullsized":"2024-03-16T10:11:40.026650+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:28:22.985440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.024225+0000","last_change":"2024-03-16T10:11:40.024225+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.024225+0000","last_undegraded":"2024-03-16T10:11:40.024225+0000","last_fullsized":"2024-03-16T10:11:40.024225+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030295+0000","last_change":"2024-03-16T10:11:39.044835+0000","last_active":"2024-03-16T10:11:40.030295+0000","last_peered":"2024-03-16T10:11:40.030295+0000","last_clean":"2024-03-16T10:11:40.030295+0000","last_became_active":"2024-03-16T10:11:39.043928+0000","last_became_peered":"2024-03-16T10:11:39.043928+0000","last_unstale":"2024-03-16T10:11:40.030295+0000","last_undegraded":"2024-03-16T10:11:40.030295+0000","last_fullsized":"2024-03-16T10:11:40.030295+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:53.886085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.035827+0000","last_change":"2024-03-16T10:11:40.034188+0000","last_active":"2024-03-16T10:11:40.035827+0000","last_peered":"2024-03-16T10:11:40.035827+0000","last_clean":"2024-03-16T10:11:40.035827+0000","last_became_active":"2024-03-16T10:11:40.033873+0000","last_became_peered":"2024-03-16T10:11:40.033873+0000","last_unstale":"2024-03-16T10:11:40.035827+0000","last_undegraded":"2024-03-16T10:11:40.035827+0000","last_fullsized":"2024-03-16T10:11:40.035827+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:32:13.692871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"29'2","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.041749+0000","last_change":"2024-03-16T10:11:39.042466+0000","last_active":"2024-03-16T10:11:40.041749+0000","last_peered":"2024-03-16T10:11:40.041749+0000","last_clean":"2024-03-16T10:11:40.041749+0000","last_became_active":"2024-03-16T10:11:39.042100+0000","last_became_peered":"2024-03-16T10:11:39.042100+0000","last_unstale":"2024-03-16T10:11:40.041749+0000","last_undegraded":"2024-03-16T10:11:40.041749+0000","last_fullsized":"2024-03-16T10:11:40.041749+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:39.788886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032944+0000","last_change":"2024-03-16T10:11:40.031195+0000","last_active":"2024-03-16T10:11:40.032944+0000","last_peered":"2024-03-16T10:11:40.032944+0000","last_clean":"2024-03-16T10:11:40.032944+0000","last_became_active":"2024-03-16T10:11:40.030881+0000","last_became_peered":"2024-03-16T10:11:40.030881+0000","last_unstale":"2024-03-16T10:11:40.032944+0000","last_undegraded":"2024-03-16T10:11:40.032944+0000","last_fullsized":"2024-03-16T10:11:40.032944+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:46:51.399892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032725+0000","last_change":"2024-03-16T10:11:39.044909+0000","last_active":"2024-03-16T10:11:40.032725+0000","last_peered":"2024-03-16T10:11:40.032725+0000","last_clean":"2024-03-16T10:11:40.032725+0000","last_became_active":"2024-03-16T10:11:39.044151+0000","last_became_peered":"2024-03-16T10:11:39.044151+0000","last_unstale":"2024-03-16T10:11:40.032725+0000","last_undegraded":"2024-03-16T10:11:40.032725+0000","last_fullsized":"2024-03-16T10:11:40.032725+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032985+0000","last_change":"2024-03-16T10:11:40.031271+0000","last_active":"2024-03-16T10:11:40.032985+0000","last_peered":"2024-03-16T10:11:40.032985+0000","last_clean":"2024-03-16T10:11:40.032985+0000","last_became_active":"2024-03-16T10:11:40.030860+0000","last_became_peered":"2024-03-16T10:11:40.030860+0000","last_unstale":"2024-03-16T10:11:40.032985+0000","last_undegraded":"2024-03-16T10:11:40.032985+0000","last_fullsized":"2024-03-16T10:11:40.032985+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:50.216565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030652+0000","last_change":"2024-03-16T10:11:39.059806+0000","last_active":"2024-03-16T10:11:40.030652+0000","last_peered":"2024-03-16T10:11:40.030652+0000","last_clean":"2024-03-16T10:11:40.030652+0000","last_became_active":"2024-03-16T10:11:39.053858+0000","last_became_peered":"2024-03-16T10:11:39.053858+0000","last_unstale":"2024-03-16T10:11:40.030652+0000","last_undegraded":"2024-03-16T10:11:40.030652+0000","last_fullsized":"2024-03-16T10:11:40.030652+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:15:51.282542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.039125+0000","last_change":"2024-03-16T10:11:40.039125+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.039125+0000","last_undegraded":"2024-03-16T10:11:40.039125+0000","last_fullsized":"2024-03-16T10:11:40.039125+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.036628+0000","last_change":"2024-03-16T10:11:39.051315+0000","last_active":"2024-03-16T10:11:40.036628+0000","last_peered":"2024-03-16T10:11:40.036628+0000","last_clean":"2024-03-16T10:11:40.036628+0000","last_became_active":"2024-03-16T10:11:39.050550+0000","last_became_peered":"2024-03-16T10:11:39.050550+0000","last_unstale":"2024-03-16T10:11:40.036628+0000","last_undegraded":"2024-03-16T10:11:40.036628+0000","last_fullsized":"2024-03-16T10:11:40.036628+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:11:55.739859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.024301+0000","last_change":"2024-03-16T10:11:40.024301+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.024301+0000","last_undegraded":"2024-03-16T10:11:40.024301+0000","last_fullsized":"2024-03-16T10:11:40.024301+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"29'2","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032987+0000","last_change":"2024-03-16T10:11:39.060338+0000","last_active":"2024-03-16T10:11:40.032987+0000","last_peered":"2024-03-16T10:11:40.032987+0000","last_clean":"2024-03-16T10:11:40.032987+0000","last_became_active":"2024-03-16T10:11:39.060040+0000","last_became_peered":"2024-03-16T10:11:39.060040+0000","last_unstale":"2024-03-16T10:11:40.032987+0000","last_undegraded":"2024-03-16T10:11:40.032987+0000","last_fullsized":"2024-03-16T10:11:40.032987+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:39.788886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.036465+0000","last_change":"2024-03-16T10:11:40.036465+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.036465+0000","last_undegraded":"2024-03-16T10:11:40.036465+0000","last_fullsized":"2024-03-16T10:11:40.036465+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027701+0000","last_change":"2024-03-16T10:11:39.042670+0000","last_active":"2024-03-16T10:11:40.027701+0000","last_peered":"2024-03-16T10:11:40.027701+0000","last_clean":"2024-03-16T10:11:40.027701+0000","last_became_active":"2024-03-16T10:11:39.042250+0000","last_became_peered":"2024-03-16T10:11:39.042250+0000","last_unstale":"2024-03-16T10:11:40.027701+0000","last_undegraded":"2024-03-16T10:11:40.027701+0000","last_fullsized":"2024-03-16T10:11:40.027701+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.036767+0000","last_change":"2024-03-16T10:11:40.035336+0000","last_active":"2024-03-16T10:11:40.036767+0000","last_peered":"2024-03-16T10:11:40.036767+0000","last_clean":"2024-03-16T10:11:40.036767+0000","last_became_active":"2024-03-16T10:11:40.035053+0000","last_became_peered":"2024-03-16T10:11:40.035053+0000","last_unstale":"2024-03-16T10:11:40.036767+0000","last_undegraded":"2024-03-16T10:11:40.036767+0000","last_fullsized":"2024-03-16T10:11:40.036767+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:54.938794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032840+0000","last_change":"2024-03-16T10:11:39.034786+0000","last_active":"2024-03-16T10:11:40.032840+0000","last_peered":"2024-03-16T10:11:40.032840+0000","last_clean":"2024-03-16T10:11:40.032840+0000","last_became_active":"2024-03-16T10:11:39.034431+0000","last_became_peered":"2024-03-16T10:11:39.034431+0000","last_unstale":"2024-03-16T10:11:40.032840+0000","last_undegraded":"2024-03-16T10:11:40.032840+0000","last_fullsized":"2024-03-16T10:11:40.032840+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.037552+0000","last_change":"2024-03-16T10:11:40.031811+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.037552+0000","last_undegraded":"2024-03-16T10:11:40.037552+0000","last_fullsized":"2024-03-16T10:11:40.037552+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.028056+0000","last_change":"2024-03-16T10:11:39.054972+0000","last_active":"2024-03-16T10:11:40.028056+0000","last_peered":"2024-03-16T10:11:40.028056+0000","last_clean":"2024-03-16T10:11:40.028056+0000","last_became_active":"2024-03-16T10:11:39.054554+0000","last_became_peered":"2024-03-16T10:11:39.054554+0000","last_unstale":"2024-03-16T10:11:40.028056+0000","last_undegraded":"2024-03-16T10:11:40.028056+0000","last_fullsized":"2024-03-16T10:11:40.028056+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:48.922456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.027157+0000","last_change":"2024-03-16T10:11:40.027157+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.027157+0000","last_undegraded":"2024-03-16T10:11:40.027157+0000","last_fullsized":"2024-03-16T10:11:40.027157+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032789+0000","last_change":"2024-03-16T10:11:39.055099+0000","last_active":"2024-03-16T10:11:40.032789+0000","last_peered":"2024-03-16T10:11:40.032789+0000","last_clean":"2024-03-16T10:11:40.032789+0000","last_became_active":"2024-03-16T10:11:39.054743+0000","last_became_peered":"2024-03-16T10:11:39.054743+0000","last_unstale":"2024-03-16T10:11:40.032789+0000","last_undegraded":"2024-03-16T10:11:40.032789+0000","last_fullsized":"2024-03-16T10:11:40.032789+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049077+0000","last_change":"2024-03-16T10:11:40.045840+0000","last_active":"2024-03-16T10:11:40.049077+0000","last_peered":"2024-03-16T10:11:40.049077+0000","last_clean":"2024-03-16T10:11:40.049077+0000","last_became_active":"2024-03-16T10:11:40.040556+0000","last_became_peered":"2024-03-16T10:11:40.040556+0000","last_unstale":"2024-03-16T10:11:40.049077+0000","last_undegraded":"2024-03-16T10:11:40.049077+0000","last_fullsized":"2024-03-16T10:11:40.049077+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:44:53.636655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.031865+0000","last_change":"2024-03-16T10:11:39.053883+0000","last_active":"2024-03-16T10:11:40.031865+0000","last_peered":"2024-03-16T10:11:40.031865+0000","last_clean":"2024-03-16T10:11:40.031865+0000","last_became_active":"2024-03-16T10:11:39.052424+0000","last_became_peered":"2024-03-16T10:11:39.052424+0000","last_unstale":"2024-03-16T10:11:40.031865+0000","last_undegraded":"2024-03-16T10:11:40.031865+0000","last_fullsized":"2024-03-16T10:11:40.031865+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:35:59.698889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049061+0000","last_change":"2024-03-16T10:11:40.049061+0000","last_active":"2024-03-16T10:11:40.049061+0000","last_peered":"2024-03-16T10:11:40.049061+0000","last_clean":"2024-03-16T10:11:40.049061+0000","last_became_active":"2024-03-16T10:11:40.048340+0000","last_became_peered":"2024-03-16T10:11:40.048340+0000","last_unstale":"2024-03-16T10:11:40.049061+0000","last_undegraded":"2024-03-16T10:11:40.049061+0000","last_fullsized":"2024-03-16T10:11:40.049061+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:21.380536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.034511+0000","last_change":"2024-03-16T10:11:39.049474+0000","last_active":"2024-03-16T10:11:40.034511+0000","last_peered":"2024-03-16T10:11:40.034511+0000","last_clean":"2024-03-16T10:11:40.034511+0000","last_became_active":"2024-03-16T10:11:39.049180+0000","last_became_peered":"2024-03-16T10:11:39.049180+0000","last_unstale":"2024-03-16T10:11:40.034511+0000","last_undegraded":"2024-03-16T10:11:40.034511+0000","last_fullsized":"2024-03-16T10:11:40.034511+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:34:23.568982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.045609+0000","last_change":"2024-03-16T10:11:40.039231+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.045609+0000","last_undegraded":"2024-03-16T10:11:40.045609+0000","last_fullsized":"2024-03-16T10:11:40.045609+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027544+0000","last_change":"2024-03-16T10:11:39.055688+0000","last_active":"2024-03-16T10:11:40.027544+0000","last_peered":"2024-03-16T10:11:40.027544+0000","last_clean":"2024-03-16T10:11:40.027544+0000","last_became_active":"2024-03-16T10:11:39.054833+0000","last_became_peered":"2024-03-16T10:11:39.054833+0000","last_unstale":"2024-03-16T10:11:40.027544+0000","last_undegraded":"2024-03-16T10:11:40.027544+0000","last_fullsized":"2024-03-16T10:11:40.027544+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":185,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045097+0000","last_change":"2024-03-16T10:10:48.248686+0000","last_active":"2024-03-16T10:11:40.045097+0000","last_peered":"2024-03-16T10:11:40.045097+0000","last_clean":"2024-03-16T10:11:40.045097+0000","last_became_active":"2024-03-16T10:10:48.243054+0000","last_became_peered":"2024-03-16T10:10:48.243054+0000","last_unstale":"2024-03-16T10:11:40.045097+0000","last_undegraded":"2024-03-16T10:11:40.045097+0000","last_fullsized":"2024-03-16T10:11:40.045097+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_clean_scrub_stamp":"2024-03-16T10:10:46.517938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:42.906473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1436,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32064,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339776,"statfs":{"total":96636764160,"available":96603930624,"internally_reserved":0,"allocated":905216,"data_stored":705976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:11:43.647 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:11:43.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb67082c640 1 Processor -- start 2024-03-16T10:11:43.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb67082c640 1 -- start start 2024-03-16T10:11:43.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb67082c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb66c067220 0x7fb66c06cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb67082c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb66c144890 con 0x7fb66c141d30 2024-03-16T10:11:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb67082c640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb66c144b70 con 0x7fb66c144300 2024-03-16T10:11:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb67082c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb66c144e50 con 0x7fb66c067220 2024-03-16T10:11:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66ad9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb66c141d30 0x7fb66c14eba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48490/0 (socket says 172.21.15.17:48490) 2024-03-16T10:11:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66ad9d640 1 -- 172.21.15.17:0/2146565780 learned_addr learned my addr 172.21.15.17:0/2146565780 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66a59c640 1 --2- 172.21.15.17:0/2146565780 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb66c067220 0x7fb66c06cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 -- 172.21.15.17:0/2146565780 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 436000589 0 0) 0x7fb66c144890 con 0x7fb66c141d30 2024-03-16T10:11:43.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 -- 172.21.15.17:0/2146565780 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb654003530 con 0x7fb66c141d30 2024-03-16T10:11:43.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 -- 172.21.15.17:0/2146565780 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3981121956 0 0) 0x7fb654003530 con 0x7fb66c141d30 2024-03-16T10:11:43.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 -- 172.21.15.17:0/2146565780 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb66c067220 msgr2=0x7fb66c06cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:43.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 --2- 172.21.15.17:0/2146565780 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb66c067220 0x7fb66c06cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 -- 172.21.15.17:0/2146565780 >> v1:172.21.15.174:6789/0 conn(0x7fb66c144300 legacy=0x7fb66c141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:43.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 -- 172.21.15.17:0/2146565780 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66c067600 con 0x7fb66c141d30 2024-03-16T10:11:43.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 -- 172.21.15.17:0/2146565780 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fb65c001680 con 0x7fb66c141d30 2024-03-16T10:11:43.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 -- 172.21.15.17:0/2146565780 >> v1:172.21.15.17:6789/0 conn(0x7fb66c141d30 legacy=0x7fb66c14eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:43.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 --2- 172.21.15.17:0/2146565780 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb66c067220 0x7fb6540039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 --2- 172.21.15.17:0/2146565780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb654003f30 0x7fb654004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 --2- 172.21.15.17:0/2146565780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654004d50 0x7fb6540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 -- 172.21.15.17:0/2146565780 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb66c144e50 con 0x7fb654003f30 2024-03-16T10:11:43.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 -- 172.21.15.17:0/2146565780 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb65400bbc0 con 0x7fb654004d50 2024-03-16T10:11:43.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66959a640 1 -- 172.21.15.17:0/2146565780 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb65400bdc0 con 0x7fb66c067220 2024-03-16T10:11:43.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb669d9b640 1 --2- 172.21.15.17:0/2146565780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb654003f30 0x7fb654004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66ad9d640 1 --2- 172.21.15.17:0/2146565780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654004d50 0x7fb6540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb66a59c640 1 --2- 172.21.15.17:0/2146565780 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb66c067220 0x7fb6540039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb669d9b640 1 -- 172.21.15.17:0/2146565780 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb66c067220 msgr2=0x7fb6540039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:43.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb669d9b640 1 --2- 172.21.15.17:0/2146565780 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb66c067220 0x7fb6540039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb669d9b640 1 -- 172.21.15.17:0/2146565780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654004d50 msgr2=0x7fb6540051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:43.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb669d9b640 1 --2- 172.21.15.17:0/2146565780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654004d50 0x7fb6540051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.804+0000 7fb669d9b640 1 -- 172.21.15.17:0/2146565780 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb654003530 con 0x7fb654003f30 2024-03-16T10:11:43.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.808+0000 7fb669d9b640 1 --2- 172.21.15.17:0/2146565780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb654003f30 0x7fb654004810 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb660001860 tx=0x7fb66001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b732f07e05d0e1f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:43.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.808+0000 7fb66959a640 1 -- 172.21.15.17:0/2146565780 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6600b0070 con 0x7fb654003f30 2024-03-16T10:11:43.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.808+0000 7fb66959a640 1 -- 172.21.15.17:0/2146565780 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6600ab600 con 0x7fb654003f30 2024-03-16T10:11:43.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.808+0000 7fb66959a640 1 -- 172.21.15.17:0/2146565780 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb660001460 con 0x7fb654003f30 2024-03-16T10:11:43.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.808+0000 7fb67082c640 1 -- 172.21.15.17:0/2146565780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb654003f30 msgr2=0x7fb654004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:43.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.808+0000 7fb67082c640 1 --2- 172.21.15.17:0/2146565780 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb654003f30 0x7fb654004810 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb660001860 tx=0x7fb66001c4b0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.808+0000 7fb66ad9d640 1 -- 172.21.15.17:0/2146565780 reap_dead start 2024-03-16T10:11:43.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.808+0000 7fb67082c640 1 -- 172.21.15.17:0/2146565780 shutdown_connections 2024-03-16T10:11:43.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.808+0000 7fb67082c640 1 -- 172.21.15.17:0/2146565780 >> 172.21.15.17:0/2146565780 conn(0x7fb66c0b9380 msgr2=0x7fb66c0b9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:43.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.808+0000 7fb67082c640 1 -- 172.21.15.17:0/2146565780 shutdown_connections 2024-03-16T10:11:43.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.808+0000 7fb67082c640 1 -- 172.21.15.17:0/2146565780 wait complete. 2024-03-16T10:11:43.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb67082c640 1 Processor -- start 2024-03-16T10:11:43.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb67082c640 1 -- start start 2024-03-16T10:11:43.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb67082c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb66c1438f0 0x7fb66c1dfe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb67082c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb66c13e910 0x7fb66c13c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb67082c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb66c13cb10 0x7fb66c14ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb67082c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb66c150440 con 0x7fb66c13cb10 2024-03-16T10:11:43.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb67082c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb66c1502c0 con 0x7fb66c13e910 2024-03-16T10:11:43.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb67082c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb66c1505c0 con 0x7fb66c1438f0 2024-03-16T10:11:43.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb66ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb66c13cb10 0x7fb66c14ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb66ad9d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb66c13cb10 0x7fb66c14ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52508/0 (socket says 172.21.15.17:52508) 2024-03-16T10:11:43.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb66ad9d640 1 -- 172.21.15.17:0/2692256420 learned_addr learned my addr 172.21.15.17:0/2692256420 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:43.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb669d9b640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb66c13e910 0x7fb66c13c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb66a59c640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb66c1438f0 0x7fb66c1dfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb66ad9d640 1 -- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb66c1438f0 msgr2=0x7fb66c1dfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:43.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb66ad9d640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb66c1438f0 0x7fb66c1dfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb66ad9d640 1 -- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb66c13e910 msgr2=0x7fb66c13c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:43.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb66ad9d640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb66c13e910 0x7fb66c13c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:43.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb66ad9d640 1 -- 172.21.15.17:0/2692256420 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb66c072320 con 0x7fb66c13cb10 2024-03-16T10:11:43.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb669d9b640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb66c13e910 0x7fb66c13c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:43.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb66ad9d640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb66c13cb10 0x7fb66c14ef90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0b7230 tx=0x7fb65c00ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:43.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb65affd640 1 -- 172.21.15.17:0/2692256420 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c019070 con 0x7fb66c13cb10 2024-03-16T10:11:43.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb65affd640 1 -- 172.21.15.17:0/2692256420 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb65c0115b0 con 0x7fb66c13cb10 2024-03-16T10:11:43.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb67082c640 1 -- 172.21.15.17:0/2692256420 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb66c13cf90 con 0x7fb66c13cb10 2024-03-16T10:11:43.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb65affd640 1 -- 172.21.15.17:0/2692256420 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65c0119a0 con 0x7fb66c13cb10 2024-03-16T10:11:43.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb67082c640 1 -- 172.21.15.17:0/2692256420 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb66c13f190 con 0x7fb66c13cb10 2024-03-16T10:11:43.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.812+0000 7fb67082c640 1 -- 172.21.15.17:0/2692256420 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb66c13f3d0 con 0x7fb66c13cb10 2024-03-16T10:11:43.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.816+0000 7fb65affd640 1 -- 172.21.15.17:0/2692256420 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fb65c01c070 con 0x7fb66c13cb10 2024-03-16T10:11:43.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.816+0000 7fb65affd640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb64c067660 0x7fb64c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:43.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.816+0000 7fb65affd640 1 -- 172.21.15.17:0/2692256420 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fb65c084b00 con 0x7fb66c13cb10 2024-03-16T10:11:43.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.816+0000 7fb66a59c640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb64c067660 0x7fb64c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:43.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.816+0000 7fb66a59c640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb64c067660 0x7fb64c069b20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb664007520 tx=0x7fb66400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:43.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.820+0000 7fb65affd640 1 -- 172.21.15.17:0/2692256420 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb66c13f3d0 con 0x7fb66c13cb10 2024-03-16T10:11:43.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.976+0000 7fb67082c640 1 -- 172.21.15.17:0/2692256420 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb66c070c40 con 0x7fb64c067660 2024-03-16T10:11:43.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.992+0000 7fb65affd640 1 -- 172.21.15.17:0/2692256420 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249410 (secure 0 0 0) 0x7fb66c070c40 con 0x7fb64c067660 2024-03-16T10:11:44.002 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:44.003 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-16T10:11:44.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.996+0000 7fb67082c640 1 -- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb64c067660 msgr2=0x7fb64c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.996+0000 7fb67082c640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb64c067660 0x7fb64c069b20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb664007520 tx=0x7fb66400b040 comp rx=0 tx=0).stop 2024-03-16T10:11:44.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.996+0000 7fb67082c640 1 -- 172.21.15.17:0/2692256420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb66c13cb10 msgr2=0x7fb66c14ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.996+0000 7fb67082c640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb66c13cb10 0x7fb66c14ef90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0b7230 tx=0x7fb65c00ead0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.996+0000 7fb67082c640 1 -- 172.21.15.17:0/2692256420 shutdown_connections 2024-03-16T10:11:44.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.996+0000 7fb67082c640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb64c067660 0x7fb64c069b20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.996+0000 7fb67082c640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb66c13cb10 0x7fb66c14ef90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.996+0000 7fb67082c640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb66c13e910 0x7fb66c13c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.996+0000 7fb67082c640 1 --2- 172.21.15.17:0/2692256420 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb66c1438f0 0x7fb66c1dfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.996+0000 7fb67082c640 1 -- 172.21.15.17:0/2692256420 >> 172.21.15.17:0/2692256420 conn(0x7fb66c0b9380 msgr2=0x7fb66c13f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:44.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.996+0000 7fb67082c640 1 -- 172.21.15.17:0/2692256420 shutdown_connections 2024-03-16T10:11:44.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:43.996+0000 7fb67082c640 1 -- 172.21.15.17:0/2692256420 wait complete. 2024-03-16T10:11:44.020 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-03-16T10:11:42.029504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591804,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":201,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236416,"kb_used_data":3592,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738304,"statfs":{"total":773094113280,"available":772852023296,"internally_reserved":0,"allocated":3678208,"data_stored":2105600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1436,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.017091"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.029409+0000","last_change":"2024-03-16T10:11:40.029409+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.029409+0000","last_undegraded":"2024-03-16T10:11:40.029409+0000","last_fullsized":"2024-03-16T10:11:40.029409+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046589+0000","last_change":"2024-03-16T10:11:40.042825+0000","last_active":"2024-03-16T10:11:40.046589+0000","last_peered":"2024-03-16T10:11:40.046589+0000","last_clean":"2024-03-16T10:11:40.046589+0000","last_became_active":"2024-03-16T10:11:40.042533+0000","last_became_peered":"2024-03-16T10:11:40.042533+0000","last_unstale":"2024-03-16T10:11:40.046589+0000","last_undegraded":"2024-03-16T10:11:40.046589+0000","last_fullsized":"2024-03-16T10:11:40.046589+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:09:41.244065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.044573+0000","last_change":"2024-03-16T10:11:40.044573+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.044573+0000","last_undegraded":"2024-03-16T10:11:40.044573+0000","last_fullsized":"2024-03-16T10:11:40.044573+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048335+0000","last_change":"2024-03-16T10:11:40.045382+0000","last_active":"2024-03-16T10:11:40.048335+0000","last_peered":"2024-03-16T10:11:40.048335+0000","last_clean":"2024-03-16T10:11:40.048335+0000","last_became_active":"2024-03-16T10:11:40.045183+0000","last_became_peered":"2024-03-16T10:11:40.045183+0000","last_unstale":"2024-03-16T10:11:40.048335+0000","last_undegraded":"2024-03-16T10:11:40.048335+0000","last_fullsized":"2024-03-16T10:11:40.048335+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:04.144054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045289+0000","last_change":"2024-03-16T10:11:40.044298+0000","last_active":"2024-03-16T10:11:40.045289+0000","last_peered":"2024-03-16T10:11:40.045289+0000","last_clean":"2024-03-16T10:11:40.045289+0000","last_became_active":"2024-03-16T10:11:40.043984+0000","last_became_peered":"2024-03-16T10:11:40.043984+0000","last_unstale":"2024-03-16T10:11:40.045289+0000","last_undegraded":"2024-03-16T10:11:40.045289+0000","last_fullsized":"2024-03-16T10:11:40.045289+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:50.672600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.028131+0000","last_change":"2024-03-16T10:11:40.028131+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.028131+0000","last_undegraded":"2024-03-16T10:11:40.028131+0000","last_fullsized":"2024-03-16T10:11:40.028131+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":["3","5"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.040072+0000","last_change":"2024-03-16T10:11:40.040072+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.040072+0000","last_undegraded":"2024-03-16T10:11:40.040072+0000","last_fullsized":"2024-03-16T10:11:40.040072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.045399+0000","last_change":"2024-03-16T10:11:40.038809+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.045399+0000","last_undegraded":"2024-03-16T10:11:40.045399+0000","last_fullsized":"2024-03-16T10:11:40.045399+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.047641+0000","last_change":"2024-03-16T10:11:40.047641+0000","last_active":"2024-03-16T10:11:40.047641+0000","last_peered":"2024-03-16T10:11:40.047641+0000","last_clean":"2024-03-16T10:11:40.047641+0000","last_became_active":"2024-03-16T10:11:40.046666+0000","last_became_peered":"2024-03-16T10:11:40.046666+0000","last_unstale":"2024-03-16T10:11:40.047641+0000","last_undegraded":"2024-03-16T10:11:40.047641+0000","last_fullsized":"2024-03-16T10:11:40.047641+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:02.965529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048493+0000","last_change":"2024-03-16T10:11:40.047611+0000","last_active":"2024-03-16T10:11:40.048493+0000","last_peered":"2024-03-16T10:11:40.048493+0000","last_clean":"2024-03-16T10:11:40.048493+0000","last_became_active":"2024-03-16T10:11:40.047371+0000","last_became_peered":"2024-03-16T10:11:40.047371+0000","last_unstale":"2024-03-16T10:11:40.048493+0000","last_undegraded":"2024-03-16T10:11:40.048493+0000","last_fullsized":"2024-03-16T10:11:40.048493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:34:24.572872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.049110+0000","last_change":"2024-03-16T10:11:40.049110+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.049110+0000","last_undegraded":"2024-03-16T10:11:40.049110+0000","last_fullsized":"2024-03-16T10:11:40.049110+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.043630+0000","last_change":"2024-03-16T10:11:40.043630+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.043630+0000","last_undegraded":"2024-03-16T10:11:40.043630+0000","last_fullsized":"2024-03-16T10:11:40.043630+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.039941+0000","last_change":"2024-03-16T10:11:40.038996+0000","last_active":"2024-03-16T10:11:40.039941+0000","last_peered":"2024-03-16T10:11:40.039941+0000","last_clean":"2024-03-16T10:11:40.039941+0000","last_became_active":"2024-03-16T10:11:40.036020+0000","last_became_peered":"2024-03-16T10:11:40.036020+0000","last_unstale":"2024-03-16T10:11:40.039941+0000","last_undegraded":"2024-03-16T10:11:40.039941+0000","last_fullsized":"2024-03-16T10:11:40.039941+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:28:23.989331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046436+0000","last_change":"2024-03-16T10:11:40.042699+0000","last_active":"2024-03-16T10:11:40.046436+0000","last_peered":"2024-03-16T10:11:40.046436+0000","last_clean":"2024-03-16T10:11:40.046436+0000","last_became_active":"2024-03-16T10:11:40.042299+0000","last_became_peered":"2024-03-16T10:11:40.042299+0000","last_unstale":"2024-03-16T10:11:40.046436+0000","last_undegraded":"2024-03-16T10:11:40.046436+0000","last_fullsized":"2024-03-16T10:11:40.046436+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:39:53.674994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.035844+0000","last_change":"2024-03-16T10:11:40.035844+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.035844+0000","last_undegraded":"2024-03-16T10:11:40.035844+0000","last_fullsized":"2024-03-16T10:11:40.035844+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.041855+0000","last_change":"2024-03-16T10:11:40.041855+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.041855+0000","last_undegraded":"2024-03-16T10:11:40.041855+0000","last_fullsized":"2024-03-16T10:11:40.041855+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.049662+0000","last_change":"2024-03-16T10:11:40.049662+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.049662+0000","last_undegraded":"2024-03-16T10:11:40.049662+0000","last_fullsized":"2024-03-16T10:11:40.049662+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049450+0000","last_change":"2024-03-16T10:11:40.046723+0000","last_active":"2024-03-16T10:11:40.049450+0000","last_peered":"2024-03-16T10:11:40.049450+0000","last_clean":"2024-03-16T10:11:40.049450+0000","last_became_active":"2024-03-16T10:11:40.043586+0000","last_became_peered":"2024-03-16T10:11:40.043586+0000","last_unstale":"2024-03-16T10:11:40.049450+0000","last_undegraded":"2024-03-16T10:11:40.049450+0000","last_fullsized":"2024-03-16T10:11:40.049450+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:06.536564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046506+0000","last_change":"2024-03-16T10:11:40.043055+0000","last_active":"2024-03-16T10:11:40.046506+0000","last_peered":"2024-03-16T10:11:40.046506+0000","last_clean":"2024-03-16T10:11:40.046506+0000","last_became_active":"2024-03-16T10:11:40.041082+0000","last_became_peered":"2024-03-16T10:11:40.041082+0000","last_unstale":"2024-03-16T10:11:40.046506+0000","last_undegraded":"2024-03-16T10:11:40.046506+0000","last_fullsized":"2024-03-16T10:11:40.046506+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:35:29.488404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.023769+0000","last_change":"2024-03-16T10:11:40.023769+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.023769+0000","last_undegraded":"2024-03-16T10:11:40.023769+0000","last_fullsized":"2024-03-16T10:11:40.023769+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046160+0000","last_change":"2024-03-16T10:11:40.046160+0000","last_active":"2024-03-16T10:11:40.046160+0000","last_peered":"2024-03-16T10:11:40.046160+0000","last_clean":"2024-03-16T10:11:40.046160+0000","last_became_active":"2024-03-16T10:11:40.045457+0000","last_became_peered":"2024-03-16T10:11:40.045457+0000","last_unstale":"2024-03-16T10:11:40.046160+0000","last_undegraded":"2024-03-16T10:11:40.046160+0000","last_fullsized":"2024-03-16T10:11:40.046160+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:58.221102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.050020+0000","last_change":"2024-03-16T10:11:40.050020+0000","last_active":"2024-03-16T10:11:40.050020+0000","last_peered":"2024-03-16T10:11:40.050020+0000","last_clean":"2024-03-16T10:11:40.050020+0000","last_became_active":"2024-03-16T10:11:40.047232+0000","last_became_peered":"2024-03-16T10:11:40.047232+0000","last_unstale":"2024-03-16T10:11:40.050020+0000","last_undegraded":"2024-03-16T10:11:40.050020+0000","last_fullsized":"2024-03-16T10:11:40.050020+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:49.926346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.027173+0000","last_change":"2024-03-16T10:11:40.027173+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.027173+0000","last_undegraded":"2024-03-16T10:11:40.027173+0000","last_fullsized":"2024-03-16T10:11:40.027173+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.050081+0000","last_change":"2024-03-16T10:11:40.041024+0000","last_active":"2024-03-16T10:11:40.050081+0000","last_peered":"2024-03-16T10:11:40.050081+0000","last_clean":"2024-03-16T10:11:40.050081+0000","last_became_active":"2024-03-16T10:11:40.040230+0000","last_became_peered":"2024-03-16T10:11:40.040230+0000","last_unstale":"2024-03-16T10:11:40.050081+0000","last_undegraded":"2024-03-16T10:11:40.050081+0000","last_fullsized":"2024-03-16T10:11:40.050081+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:19.819589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.040254+0000","last_change":"2024-03-16T10:11:40.036983+0000","last_active":"2024-03-16T10:11:40.040254+0000","last_peered":"2024-03-16T10:11:40.040254+0000","last_clean":"2024-03-16T10:11:40.040254+0000","last_became_active":"2024-03-16T10:11:40.036701+0000","last_became_peered":"2024-03-16T10:11:40.036701+0000","last_unstale":"2024-03-16T10:11:40.040254+0000","last_undegraded":"2024-03-16T10:11:40.040254+0000","last_fullsized":"2024-03-16T10:11:40.040254+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:54.938794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.028496+0000","last_change":"2024-03-16T10:11:40.028496+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.028496+0000","last_undegraded":"2024-03-16T10:11:40.028496+0000","last_fullsized":"2024-03-16T10:11:40.028496+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048445+0000","last_change":"2024-03-16T10:11:40.046735+0000","last_active":"2024-03-16T10:11:40.048445+0000","last_peered":"2024-03-16T10:11:40.048445+0000","last_clean":"2024-03-16T10:11:40.048445+0000","last_became_active":"2024-03-16T10:11:40.046202+0000","last_became_peered":"2024-03-16T10:11:40.046202+0000","last_unstale":"2024-03-16T10:11:40.048445+0000","last_undegraded":"2024-03-16T10:11:40.048445+0000","last_fullsized":"2024-03-16T10:11:40.048445+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:48.954207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.042755+0000","last_change":"2024-03-16T10:11:40.042755+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.042755+0000","last_undegraded":"2024-03-16T10:11:40.042755+0000","last_fullsized":"2024-03-16T10:11:40.042755+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.067258+0000","last_change":"2024-03-16T10:11:40.067258+0000","last_active":"2024-03-16T10:11:40.067258+0000","last_peered":"2024-03-16T10:11:40.067258+0000","last_clean":"2024-03-16T10:11:40.067258+0000","last_became_active":"2024-03-16T10:11:40.066423+0000","last_became_peered":"2024-03-16T10:11:40.066423+0000","last_unstale":"2024-03-16T10:11:40.067258+0000","last_undegraded":"2024-03-16T10:11:40.067258+0000","last_fullsized":"2024-03-16T10:11:40.067258+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:04.144054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.041848+0000","last_change":"2024-03-16T10:11:40.041848+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.041848+0000","last_undegraded":"2024-03-16T10:11:40.041848+0000","last_fullsized":"2024-03-16T10:11:40.041848+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.034714+0000","last_change":"2024-03-16T10:11:40.028766+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.034714+0000","last_undegraded":"2024-03-16T10:11:40.034714+0000","last_fullsized":"2024-03-16T10:11:40.034714+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":["6","2"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.039748+0000","last_change":"2024-03-16T10:11:40.039748+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.039748+0000","last_undegraded":"2024-03-16T10:11:40.039748+0000","last_fullsized":"2024-03-16T10:11:40.039748+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.027172+0000","last_change":"2024-03-16T10:11:40.027172+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.027172+0000","last_undegraded":"2024-03-16T10:11:40.027172+0000","last_fullsized":"2024-03-16T10:11:40.027172+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"29'4","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030235+0000","last_change":"2024-03-16T10:11:39.061998+0000","last_active":"2024-03-16T10:11:40.030235+0000","last_peered":"2024-03-16T10:11:40.030235+0000","last_clean":"2024-03-16T10:11:40.030235+0000","last_became_active":"2024-03-16T10:11:39.061249+0000","last_became_peered":"2024-03-16T10:11:39.061249+0000","last_unstale":"2024-03-16T10:11:40.030235+0000","last_undegraded":"2024-03-16T10:11:40.030235+0000","last_fullsized":"2024-03-16T10:11:40.030235+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.061676+0000","last_change":"2024-03-16T10:11:40.061676+0000","last_active":"2024-03-16T10:11:40.061676+0000","last_peered":"2024-03-16T10:11:40.061676+0000","last_clean":"2024-03-16T10:11:40.061676+0000","last_became_active":"2024-03-16T10:11:40.060865+0000","last_became_peered":"2024-03-16T10:11:40.060865+0000","last_unstale":"2024-03-16T10:11:40.061676+0000","last_undegraded":"2024-03-16T10:11:40.061676+0000","last_fullsized":"2024-03-16T10:11:40.061676+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:40.792776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027530+0000","last_change":"2024-03-16T10:11:39.048117+0000","last_active":"2024-03-16T10:11:40.027530+0000","last_peered":"2024-03-16T10:11:40.027530+0000","last_clean":"2024-03-16T10:11:40.027530+0000","last_became_active":"2024-03-16T10:11:39.047798+0000","last_became_peered":"2024-03-16T10:11:39.047798+0000","last_unstale":"2024-03-16T10:11:40.027530+0000","last_undegraded":"2024-03-16T10:11:40.027530+0000","last_fullsized":"2024-03-16T10:11:40.027530+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.062870+0000","last_change":"2024-03-16T10:11:40.062870+0000","last_active":"2024-03-16T10:11:40.062870+0000","last_peered":"2024-03-16T10:11:40.062870+0000","last_clean":"2024-03-16T10:11:40.062870+0000","last_became_active":"2024-03-16T10:11:40.054629+0000","last_became_peered":"2024-03-16T10:11:40.054629+0000","last_unstale":"2024-03-16T10:11:40.062870+0000","last_undegraded":"2024-03-16T10:11:40.062870+0000","last_fullsized":"2024-03-16T10:11:40.062870+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:40.792776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"29'2","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049288+0000","last_change":"2024-03-16T10:11:39.038358+0000","last_active":"2024-03-16T10:11:40.049288+0000","last_peered":"2024-03-16T10:11:40.049288+0000","last_clean":"2024-03-16T10:11:40.049288+0000","last_became_active":"2024-03-16T10:11:39.038055+0000","last_became_peered":"2024-03-16T10:11:39.038055+0000","last_unstale":"2024-03-16T10:11:40.049288+0000","last_undegraded":"2024-03-16T10:11:40.049288+0000","last_fullsized":"2024-03-16T10:11:40.049288+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:11:55.739859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048424+0000","last_change":"2024-03-16T10:11:40.046475+0000","last_active":"2024-03-16T10:11:40.048424+0000","last_peered":"2024-03-16T10:11:40.048424+0000","last_clean":"2024-03-16T10:11:40.048424+0000","last_became_active":"2024-03-16T10:11:40.045876+0000","last_became_peered":"2024-03-16T10:11:40.045876+0000","last_unstale":"2024-03-16T10:11:40.048424+0000","last_undegraded":"2024-03-16T10:11:40.048424+0000","last_fullsized":"2024-03-16T10:11:40.048424+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:35:43.212538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"29'2","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045706+0000","last_change":"2024-03-16T10:11:39.030352+0000","last_active":"2024-03-16T10:11:40.045706+0000","last_peered":"2024-03-16T10:11:40.045706+0000","last_clean":"2024-03-16T10:11:40.045706+0000","last_became_active":"2024-03-16T10:11:39.029031+0000","last_became_peered":"2024-03-16T10:11:39.029031+0000","last_unstale":"2024-03-16T10:11:40.045706+0000","last_undegraded":"2024-03-16T10:11:40.045706+0000","last_fullsized":"2024-03-16T10:11:40.045706+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:07:13.816782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.041163+0000","last_change":"2024-03-16T10:11:40.041163+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.041163+0000","last_undegraded":"2024-03-16T10:11:40.041163+0000","last_fullsized":"2024-03-16T10:11:40.041163+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032683+0000","last_change":"2024-03-16T10:11:39.061267+0000","last_active":"2024-03-16T10:11:40.032683+0000","last_peered":"2024-03-16T10:11:40.032683+0000","last_clean":"2024-03-16T10:11:40.032683+0000","last_became_active":"2024-03-16T10:11:39.060323+0000","last_became_peered":"2024-03-16T10:11:39.060323+0000","last_unstale":"2024-03-16T10:11:40.032683+0000","last_undegraded":"2024-03-16T10:11:40.032683+0000","last_fullsized":"2024-03-16T10:11:40.032683+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:33:59.939254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.040415+0000","last_change":"2024-03-16T10:11:40.040415+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.040415+0000","last_undegraded":"2024-03-16T10:11:40.040415+0000","last_fullsized":"2024-03-16T10:11:40.040415+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027685+0000","last_change":"2024-03-16T10:11:39.061259+0000","last_active":"2024-03-16T10:11:40.027685+0000","last_peered":"2024-03-16T10:11:40.027685+0000","last_clean":"2024-03-16T10:11:40.027685+0000","last_became_active":"2024-03-16T10:11:39.060484+0000","last_became_peered":"2024-03-16T10:11:39.060484+0000","last_unstale":"2024-03-16T10:11:40.027685+0000","last_undegraded":"2024-03-16T10:11:40.027685+0000","last_fullsized":"2024-03-16T10:11:40.027685+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:54.254670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":6,"reported_epoch":29,"state":"active","last_fresh":"2024-03-16T10:11:40.070794+0000","last_change":"2024-03-16T10:11:40.070642+0000","last_active":"2024-03-16T10:11:40.070794+0000","last_peered":"2024-03-16T10:11:40.070794+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"2024-03-16T10:11:40.070642+0000","last_became_peered":"2024-03-16T10:11:40.070642+0000","last_unstale":"2024-03-16T10:11:40.070794+0000","last_undegraded":"2024-03-16T10:11:40.070794+0000","last_fullsized":"2024-03-16T10:11:40.070794+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.034336+0000","last_change":"2024-03-16T10:11:39.033585+0000","last_active":"2024-03-16T10:11:40.034336+0000","last_peered":"2024-03-16T10:11:40.034336+0000","last_clean":"2024-03-16T10:11:40.034336+0000","last_became_active":"2024-03-16T10:11:39.033131+0000","last_became_peered":"2024-03-16T10:11:39.033131+0000","last_unstale":"2024-03-16T10:11:40.034336+0000","last_undegraded":"2024-03-16T10:11:40.034336+0000","last_fullsized":"2024-03-16T10:11:40.034336+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:35:42.208648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.029382+0000","last_change":"2024-03-16T10:11:40.029382+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.029382+0000","last_undegraded":"2024-03-16T10:11:40.029382+0000","last_fullsized":"2024-03-16T10:11:40.029382+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.026886+0000","last_change":"2024-03-16T10:11:39.053374+0000","last_active":"2024-03-16T10:11:40.026886+0000","last_peered":"2024-03-16T10:11:40.026886+0000","last_clean":"2024-03-16T10:11:40.026886+0000","last_became_active":"2024-03-16T10:11:39.051683+0000","last_became_peered":"2024-03-16T10:11:39.051683+0000","last_unstale":"2024-03-16T10:11:40.026886+0000","last_undegraded":"2024-03-16T10:11:40.026886+0000","last_fullsized":"2024-03-16T10:11:40.026886+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:49.668710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.046622+0000","last_change":"2024-03-16T10:11:40.046622+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.046622+0000","last_undegraded":"2024-03-16T10:11:40.046622+0000","last_fullsized":"2024-03-16T10:11:40.046622+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.035769+0000","last_change":"2024-03-16T10:11:39.050941+0000","last_active":"2024-03-16T10:11:40.035769+0000","last_peered":"2024-03-16T10:11:40.035769+0000","last_clean":"2024-03-16T10:11:40.035769+0000","last_became_active":"2024-03-16T10:11:39.050659+0000","last_became_peered":"2024-03-16T10:11:39.050659+0000","last_unstale":"2024-03-16T10:11:40.035769+0000","last_undegraded":"2024-03-16T10:11:40.035769+0000","last_fullsized":"2024-03-16T10:11:40.035769+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:08.229218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046403+0000","last_change":"2024-03-16T10:11:40.043035+0000","last_active":"2024-03-16T10:11:40.046403+0000","last_peered":"2024-03-16T10:11:40.046403+0000","last_clean":"2024-03-16T10:11:40.046403+0000","last_became_active":"2024-03-16T10:11:40.040585+0000","last_became_peered":"2024-03-16T10:11:40.040585+0000","last_unstale":"2024-03-16T10:11:40.046403+0000","last_undegraded":"2024-03-16T10:11:40.046403+0000","last_fullsized":"2024-03-16T10:11:40.046403+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:36:55.327282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027611+0000","last_change":"2024-03-16T10:11:39.045927+0000","last_active":"2024-03-16T10:11:40.027611+0000","last_peered":"2024-03-16T10:11:40.027611+0000","last_clean":"2024-03-16T10:11:40.027611+0000","last_became_active":"2024-03-16T10:11:39.045502+0000","last_became_peered":"2024-03-16T10:11:39.045502+0000","last_unstale":"2024-03-16T10:11:40.027611+0000","last_undegraded":"2024-03-16T10:11:40.027611+0000","last_fullsized":"2024-03-16T10:11:40.027611+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049439+0000","last_change":"2024-03-16T10:11:40.041235+0000","last_active":"2024-03-16T10:11:40.049439+0000","last_peered":"2024-03-16T10:11:40.049439+0000","last_clean":"2024-03-16T10:11:40.049439+0000","last_became_active":"2024-03-16T10:11:40.040856+0000","last_became_peered":"2024-03-16T10:11:40.040856+0000","last_unstale":"2024-03-16T10:11:40.049439+0000","last_undegraded":"2024-03-16T10:11:40.049439+0000","last_fullsized":"2024-03-16T10:11:40.049439+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:38:40.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"29'3","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046332+0000","last_change":"2024-03-16T10:11:39.052193+0000","last_active":"2024-03-16T10:11:40.046332+0000","last_peered":"2024-03-16T10:11:40.046332+0000","last_clean":"2024-03-16T10:11:40.046332+0000","last_became_active":"2024-03-16T10:11:39.050633+0000","last_became_peered":"2024-03-16T10:11:39.050633+0000","last_unstale":"2024-03-16T10:11:40.046332+0000","last_undegraded":"2024-03-16T10:11:40.046332+0000","last_fullsized":"2024-03-16T10:11:40.046332+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:08.229218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.045474+0000","last_change":"2024-03-16T10:11:40.045474+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.045474+0000","last_undegraded":"2024-03-16T10:11:40.045474+0000","last_fullsized":"2024-03-16T10:11:40.045474+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"29'5","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.053253+0000","last_change":"2024-03-16T10:11:39.042282+0000","last_active":"2024-03-16T10:11:40.053253+0000","last_peered":"2024-03-16T10:11:40.053253+0000","last_clean":"2024-03-16T10:11:40.053253+0000","last_became_active":"2024-03-16T10:11:39.041924+0000","last_became_peered":"2024-03-16T10:11:39.041924+0000","last_unstale":"2024-03-16T10:11:40.053253+0000","last_undegraded":"2024-03-16T10:11:40.053253+0000","last_fullsized":"2024-03-16T10:11:40.053253+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:18.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.029679+0000","last_change":"2024-03-16T10:11:40.029679+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.029679+0000","last_undegraded":"2024-03-16T10:11:40.029679+0000","last_fullsized":"2024-03-16T10:11:40.029679+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.069420+0000","last_change":"2024-03-16T10:11:39.042217+0000","last_active":"2024-03-16T10:11:40.069420+0000","last_peered":"2024-03-16T10:11:40.069420+0000","last_clean":"2024-03-16T10:11:40.069420+0000","last_became_active":"2024-03-16T10:11:39.041514+0000","last_became_peered":"2024-03-16T10:11:39.041514+0000","last_unstale":"2024-03-16T10:11:40.069420+0000","last_undegraded":"2024-03-16T10:11:40.069420+0000","last_fullsized":"2024-03-16T10:11:40.069420+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:12:12.889055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049639+0000","last_change":"2024-03-16T10:11:40.048743+0000","last_active":"2024-03-16T10:11:40.049639+0000","last_peered":"2024-03-16T10:11:40.049639+0000","last_clean":"2024-03-16T10:11:40.049639+0000","last_became_active":"2024-03-16T10:11:40.047008+0000","last_became_peered":"2024-03-16T10:11:40.047008+0000","last_unstale":"2024-03-16T10:11:40.049639+0000","last_undegraded":"2024-03-16T10:11:40.049639+0000","last_fullsized":"2024-03-16T10:11:40.049639+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:56:04.173993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032632+0000","last_change":"2024-03-16T10:11:39.047491+0000","last_active":"2024-03-16T10:11:40.032632+0000","last_peered":"2024-03-16T10:11:40.032632+0000","last_clean":"2024-03-16T10:11:40.032632+0000","last_became_active":"2024-03-16T10:11:39.047085+0000","last_became_peered":"2024-03-16T10:11:39.047085+0000","last_unstale":"2024-03-16T10:11:40.032632+0000","last_undegraded":"2024-03-16T10:11:40.032632+0000","last_fullsized":"2024-03-16T10:11:40.032632+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.040610+0000","last_change":"2024-03-16T10:11:40.040610+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.040610+0000","last_undegraded":"2024-03-16T10:11:40.040610+0000","last_fullsized":"2024-03-16T10:11:40.040610+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.034687+0000","last_change":"2024-03-16T10:11:39.053564+0000","last_active":"2024-03-16T10:11:40.034687+0000","last_peered":"2024-03-16T10:11:40.034687+0000","last_clean":"2024-03-16T10:11:40.034687+0000","last_became_active":"2024-03-16T10:11:39.053183+0000","last_became_peered":"2024-03-16T10:11:39.053183+0000","last_unstale":"2024-03-16T10:11:40.034687+0000","last_undegraded":"2024-03-16T10:11:40.034687+0000","last_fullsized":"2024-03-16T10:11:40.034687+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:45:20.816477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":6,"reported_epoch":29,"state":"active","last_fresh":"2024-03-16T10:11:40.058574+0000","last_change":"2024-03-16T10:11:40.058392+0000","last_active":"2024-03-16T10:11:40.058574+0000","last_peered":"2024-03-16T10:11:40.058574+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"2024-03-16T10:11:40.058392+0000","last_became_peered":"2024-03-16T10:11:40.058392+0000","last_unstale":"2024-03-16T10:11:40.058574+0000","last_undegraded":"2024-03-16T10:11:40.058574+0000","last_fullsized":"2024-03-16T10:11:40.058574+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"29'8","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.053703+0000","last_change":"2024-03-16T10:11:39.035694+0000","last_active":"2024-03-16T10:11:40.053703+0000","last_peered":"2024-03-16T10:11:40.053703+0000","last_clean":"2024-03-16T10:11:40.053703+0000","last_became_active":"2024-03-16T10:11:39.035382+0000","last_became_peered":"2024-03-16T10:11:39.035382+0000","last_unstale":"2024-03-16T10:11:40.053703+0000","last_undegraded":"2024-03-16T10:11:40.053703+0000","last_fullsized":"2024-03-16T10:11:40.053703+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:47.950316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.064246+0000","last_change":"2024-03-16T10:11:40.064246+0000","last_active":"2024-03-16T10:11:40.064246+0000","last_peered":"2024-03-16T10:11:40.064246+0000","last_clean":"2024-03-16T10:11:40.064246+0000","last_became_active":"2024-03-16T10:11:40.063529+0000","last_became_peered":"2024-03-16T10:11:40.063529+0000","last_unstale":"2024-03-16T10:11:40.064246+0000","last_undegraded":"2024-03-16T10:11:40.064246+0000","last_fullsized":"2024-03-16T10:11:40.064246+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:54.938794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"29'1","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045024+0000","last_change":"2024-03-16T10:11:39.046131+0000","last_active":"2024-03-16T10:11:40.045024+0000","last_peered":"2024-03-16T10:11:40.045024+0000","last_clean":"2024-03-16T10:11:40.045024+0000","last_became_active":"2024-03-16T10:11:39.045785+0000","last_became_peered":"2024-03-16T10:11:39.045785+0000","last_unstale":"2024-03-16T10:11:40.045024+0000","last_undegraded":"2024-03-16T10:11:40.045024+0000","last_fullsized":"2024-03-16T10:11:40.045024+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:48.922456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.037470+0000","last_change":"2024-03-16T10:11:40.037470+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.037470+0000","last_undegraded":"2024-03-16T10:11:40.037470+0000","last_fullsized":"2024-03-16T10:11:40.037470+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"29'2","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030277+0000","last_change":"2024-03-16T10:11:39.051711+0000","last_active":"2024-03-16T10:11:40.030277+0000","last_peered":"2024-03-16T10:11:40.030277+0000","last_clean":"2024-03-16T10:11:40.030277+0000","last_became_active":"2024-03-16T10:11:39.050887+0000","last_became_peered":"2024-03-16T10:11:39.050887+0000","last_unstale":"2024-03-16T10:11:40.030277+0000","last_undegraded":"2024-03-16T10:11:40.030277+0000","last_fullsized":"2024-03-16T10:11:40.030277+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:03:20.074611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.031547+0000","last_change":"2024-03-16T10:11:40.031547+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.031547+0000","last_undegraded":"2024-03-16T10:11:40.031547+0000","last_fullsized":"2024-03-16T10:11:40.031547+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.026650+0000","last_change":"2024-03-16T10:11:39.059183+0000","last_active":"2024-03-16T10:11:40.026650+0000","last_peered":"2024-03-16T10:11:40.026650+0000","last_clean":"2024-03-16T10:11:40.026650+0000","last_became_active":"2024-03-16T10:11:39.052311+0000","last_became_peered":"2024-03-16T10:11:39.052311+0000","last_unstale":"2024-03-16T10:11:40.026650+0000","last_undegraded":"2024-03-16T10:11:40.026650+0000","last_fullsized":"2024-03-16T10:11:40.026650+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:28:22.985440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.024225+0000","last_change":"2024-03-16T10:11:40.024225+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.024225+0000","last_undegraded":"2024-03-16T10:11:40.024225+0000","last_fullsized":"2024-03-16T10:11:40.024225+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030295+0000","last_change":"2024-03-16T10:11:39.044835+0000","last_active":"2024-03-16T10:11:40.030295+0000","last_peered":"2024-03-16T10:11:40.030295+0000","last_clean":"2024-03-16T10:11:40.030295+0000","last_became_active":"2024-03-16T10:11:39.043928+0000","last_became_peered":"2024-03-16T10:11:39.043928+0000","last_unstale":"2024-03-16T10:11:40.030295+0000","last_undegraded":"2024-03-16T10:11:40.030295+0000","last_fullsized":"2024-03-16T10:11:40.030295+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:53.886085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.035827+0000","last_change":"2024-03-16T10:11:40.034188+0000","last_active":"2024-03-16T10:11:40.035827+0000","last_peered":"2024-03-16T10:11:40.035827+0000","last_clean":"2024-03-16T10:11:40.035827+0000","last_became_active":"2024-03-16T10:11:40.033873+0000","last_became_peered":"2024-03-16T10:11:40.033873+0000","last_unstale":"2024-03-16T10:11:40.035827+0000","last_undegraded":"2024-03-16T10:11:40.035827+0000","last_fullsized":"2024-03-16T10:11:40.035827+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:32:13.692871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"29'2","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.041749+0000","last_change":"2024-03-16T10:11:39.042466+0000","last_active":"2024-03-16T10:11:40.041749+0000","last_peered":"2024-03-16T10:11:40.041749+0000","last_clean":"2024-03-16T10:11:40.041749+0000","last_became_active":"2024-03-16T10:11:39.042100+0000","last_became_peered":"2024-03-16T10:11:39.042100+0000","last_unstale":"2024-03-16T10:11:40.041749+0000","last_undegraded":"2024-03-16T10:11:40.041749+0000","last_fullsized":"2024-03-16T10:11:40.041749+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:39.788886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032944+0000","last_change":"2024-03-16T10:11:40.031195+0000","last_active":"2024-03-16T10:11:40.032944+0000","last_peered":"2024-03-16T10:11:40.032944+0000","last_clean":"2024-03-16T10:11:40.032944+0000","last_became_active":"2024-03-16T10:11:40.030881+0000","last_became_peered":"2024-03-16T10:11:40.030881+0000","last_unstale":"2024-03-16T10:11:40.032944+0000","last_undegraded":"2024-03-16T10:11:40.032944+0000","last_fullsized":"2024-03-16T10:11:40.032944+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:46:51.399892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032725+0000","last_change":"2024-03-16T10:11:39.044909+0000","last_active":"2024-03-16T10:11:40.032725+0000","last_peered":"2024-03-16T10:11:40.032725+0000","last_clean":"2024-03-16T10:11:40.032725+0000","last_became_active":"2024-03-16T10:11:39.044151+0000","last_became_peered":"2024-03-16T10:11:39.044151+0000","last_unstale":"2024-03-16T10:11:40.032725+0000","last_undegraded":"2024-03-16T10:11:40.032725+0000","last_fullsized":"2024-03-16T10:11:40.032725+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032985+0000","last_change":"2024-03-16T10:11:40.031271+0000","last_active":"2024-03-16T10:11:40.032985+0000","last_peered":"2024-03-16T10:11:40.032985+0000","last_clean":"2024-03-16T10:11:40.032985+0000","last_became_active":"2024-03-16T10:11:40.030860+0000","last_became_peered":"2024-03-16T10:11:40.030860+0000","last_unstale":"2024-03-16T10:11:40.032985+0000","last_undegraded":"2024-03-16T10:11:40.032985+0000","last_fullsized":"2024-03-16T10:11:40.032985+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:50.216565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030652+0000","last_change":"2024-03-16T10:11:39.059806+0000","last_active":"2024-03-16T10:11:40.030652+0000","last_peered":"2024-03-16T10:11:40.030652+0000","last_clean":"2024-03-16T10:11:40.030652+0000","last_became_active":"2024-03-16T10:11:39.053858+0000","last_became_peered":"2024-03-16T10:11:39.053858+0000","last_unstale":"2024-03-16T10:11:40.030652+0000","last_undegraded":"2024-03-16T10:11:40.030652+0000","last_fullsized":"2024-03-16T10:11:40.030652+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:15:51.282542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.039125+0000","last_change":"2024-03-16T10:11:40.039125+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.039125+0000","last_undegraded":"2024-03-16T10:11:40.039125+0000","last_fullsized":"2024-03-16T10:11:40.039125+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.036628+0000","last_change":"2024-03-16T10:11:39.051315+0000","last_active":"2024-03-16T10:11:40.036628+0000","last_peered":"2024-03-16T10:11:40.036628+0000","last_clean":"2024-03-16T10:11:40.036628+0000","last_became_active":"2024-03-16T10:11:39.050550+0000","last_became_peered":"2024-03-16T10:11:39.050550+0000","last_unstale":"2024-03-16T10:11:40.036628+0000","last_undegraded":"2024-03-16T10:11:40.036628+0000","last_fullsized":"2024-03-16T10:11:40.036628+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:11:55.739859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.024301+0000","last_change":"2024-03-16T10:11:40.024301+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.024301+0000","last_undegraded":"2024-03-16T10:11:40.024301+0000","last_fullsized":"2024-03-16T10:11:40.024301+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"29'2","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032987+0000","last_change":"2024-03-16T10:11:39.060338+0000","last_active":"2024-03-16T10:11:40.032987+0000","last_peered":"2024-03-16T10:11:40.032987+0000","last_clean":"2024-03-16T10:11:40.032987+0000","last_became_active":"2024-03-16T10:11:39.060040+0000","last_became_peered":"2024-03-16T10:11:39.060040+0000","last_unstale":"2024-03-16T10:11:40.032987+0000","last_undegraded":"2024-03-16T10:11:40.032987+0000","last_fullsized":"2024-03-16T10:11:40.032987+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:39.788886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.036465+0000","last_change":"2024-03-16T10:11:40.036465+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.036465+0000","last_undegraded":"2024-03-16T10:11:40.036465+0000","last_fullsized":"2024-03-16T10:11:40.036465+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027701+0000","last_change":"2024-03-16T10:11:39.042670+0000","last_active":"2024-03-16T10:11:40.027701+0000","last_peered":"2024-03-16T10:11:40.027701+0000","last_clean":"2024-03-16T10:11:40.027701+0000","last_became_active":"2024-03-16T10:11:39.042250+0000","last_became_peered":"2024-03-16T10:11:39.042250+0000","last_unstale":"2024-03-16T10:11:40.027701+0000","last_undegraded":"2024-03-16T10:11:40.027701+0000","last_fullsized":"2024-03-16T10:11:40.027701+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.036767+0000","last_change":"2024-03-16T10:11:40.035336+0000","last_active":"2024-03-16T10:11:40.036767+0000","last_peered":"2024-03-16T10:11:40.036767+0000","last_clean":"2024-03-16T10:11:40.036767+0000","last_became_active":"2024-03-16T10:11:40.035053+0000","last_became_peered":"2024-03-16T10:11:40.035053+0000","last_unstale":"2024-03-16T10:11:40.036767+0000","last_undegraded":"2024-03-16T10:11:40.036767+0000","last_fullsized":"2024-03-16T10:11:40.036767+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:54.938794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032840+0000","last_change":"2024-03-16T10:11:39.034786+0000","last_active":"2024-03-16T10:11:40.032840+0000","last_peered":"2024-03-16T10:11:40.032840+0000","last_clean":"2024-03-16T10:11:40.032840+0000","last_became_active":"2024-03-16T10:11:39.034431+0000","last_became_peered":"2024-03-16T10:11:39.034431+0000","last_unstale":"2024-03-16T10:11:40.032840+0000","last_undegraded":"2024-03-16T10:11:40.032840+0000","last_fullsized":"2024-03-16T10:11:40.032840+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.037552+0000","last_change":"2024-03-16T10:11:40.031811+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.037552+0000","last_undegraded":"2024-03-16T10:11:40.037552+0000","last_fullsized":"2024-03-16T10:11:40.037552+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.028056+0000","last_change":"2024-03-16T10:11:39.054972+0000","last_active":"2024-03-16T10:11:40.028056+0000","last_peered":"2024-03-16T10:11:40.028056+0000","last_clean":"2024-03-16T10:11:40.028056+0000","last_became_active":"2024-03-16T10:11:39.054554+0000","last_became_peered":"2024-03-16T10:11:39.054554+0000","last_unstale":"2024-03-16T10:11:40.028056+0000","last_undegraded":"2024-03-16T10:11:40.028056+0000","last_fullsized":"2024-03-16T10:11:40.028056+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:48.922456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.027157+0000","last_change":"2024-03-16T10:11:40.027157+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.027157+0000","last_undegraded":"2024-03-16T10:11:40.027157+0000","last_fullsized":"2024-03-16T10:11:40.027157+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032789+0000","last_change":"2024-03-16T10:11:39.055099+0000","last_active":"2024-03-16T10:11:40.032789+0000","last_peered":"2024-03-16T10:11:40.032789+0000","last_clean":"2024-03-16T10:11:40.032789+0000","last_became_active":"2024-03-16T10:11:39.054743+0000","last_became_peered":"2024-03-16T10:11:39.054743+0000","last_unstale":"2024-03-16T10:11:40.032789+0000","last_undegraded":"2024-03-16T10:11:40.032789+0000","last_fullsized":"2024-03-16T10:11:40.032789+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049077+0000","last_change":"2024-03-16T10:11:40.045840+0000","last_active":"2024-03-16T10:11:40.049077+0000","last_peered":"2024-03-16T10:11:40.049077+0000","last_clean":"2024-03-16T10:11:40.049077+0000","last_became_active":"2024-03-16T10:11:40.040556+0000","last_became_peered":"2024-03-16T10:11:40.040556+0000","last_unstale":"2024-03-16T10:11:40.049077+0000","last_undegraded":"2024-03-16T10:11:40.049077+0000","last_fullsized":"2024-03-16T10:11:40.049077+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:44:53.636655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.031865+0000","last_change":"2024-03-16T10:11:39.053883+0000","last_active":"2024-03-16T10:11:40.031865+0000","last_peered":"2024-03-16T10:11:40.031865+0000","last_clean":"2024-03-16T10:11:40.031865+0000","last_became_active":"2024-03-16T10:11:39.052424+0000","last_became_peered":"2024-03-16T10:11:39.052424+0000","last_unstale":"2024-03-16T10:11:40.031865+0000","last_undegraded":"2024-03-16T10:11:40.031865+0000","last_fullsized":"2024-03-16T10:11:40.031865+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:35:59.698889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049061+0000","last_change":"2024-03-16T10:11:40.049061+0000","last_active":"2024-03-16T10:11:40.049061+0000","last_peered":"2024-03-16T10:11:40.049061+0000","last_clean":"2024-03-16T10:11:40.049061+0000","last_became_active":"2024-03-16T10:11:40.048340+0000","last_became_peered":"2024-03-16T10:11:40.048340+0000","last_unstale":"2024-03-16T10:11:40.049061+0000","last_undegraded":"2024-03-16T10:11:40.049061+0000","last_fullsized":"2024-03-16T10:11:40.049061+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:21.380536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.034511+0000","last_change":"2024-03-16T10:11:39.049474+0000","last_active":"2024-03-16T10:11:40.034511+0000","last_peered":"2024-03-16T10:11:40.034511+0000","last_clean":"2024-03-16T10:11:40.034511+0000","last_became_active":"2024-03-16T10:11:39.049180+0000","last_became_peered":"2024-03-16T10:11:39.049180+0000","last_unstale":"2024-03-16T10:11:40.034511+0000","last_undegraded":"2024-03-16T10:11:40.034511+0000","last_fullsized":"2024-03-16T10:11:40.034511+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:34:23.568982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.045609+0000","last_change":"2024-03-16T10:11:40.039231+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.045609+0000","last_undegraded":"2024-03-16T10:11:40.045609+0000","last_fullsized":"2024-03-16T10:11:40.045609+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027544+0000","last_change":"2024-03-16T10:11:39.055688+0000","last_active":"2024-03-16T10:11:40.027544+0000","last_peered":"2024-03-16T10:11:40.027544+0000","last_clean":"2024-03-16T10:11:40.027544+0000","last_became_active":"2024-03-16T10:11:39.054833+0000","last_became_peered":"2024-03-16T10:11:39.054833+0000","last_unstale":"2024-03-16T10:11:40.027544+0000","last_undegraded":"2024-03-16T10:11:40.027544+0000","last_fullsized":"2024-03-16T10:11:40.027544+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":185,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045097+0000","last_change":"2024-03-16T10:10:48.248686+0000","last_active":"2024-03-16T10:11:40.045097+0000","last_peered":"2024-03-16T10:11:40.045097+0000","last_clean":"2024-03-16T10:11:40.045097+0000","last_became_active":"2024-03-16T10:10:48.243054+0000","last_became_peered":"2024-03-16T10:10:48.243054+0000","last_unstale":"2024-03-16T10:11:40.045097+0000","last_undegraded":"2024-03-16T10:11:40.045097+0000","last_fullsized":"2024-03-16T10:11:40.045097+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_clean_scrub_stamp":"2024-03-16T10:10:46.517938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:42.906473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1436,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32064,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339776,"statfs":{"total":96636764160,"available":96603930624,"internally_reserved":0,"allocated":905216,"data_stored":705976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:11:44.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-03-16T10:11:44.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.029409+0000', 'last_change': '2024-03-16T10:11:40.029409+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.029409+0000', 'last_undegraded': '2024-03-16T10:11:40.029409+0000', 'last_fullsized': '2024-03-16T10:11:40.029409+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': ['6', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-16T10:11:44.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-03-16T10:11:44.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.044573+0000', 'last_change': '2024-03-16T10:11:40.044573+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.044573+0000', 'last_undegraded': '2024-03-16T10:11:40.044573+0000', 'last_fullsized': '2024-03-16T10:11:40.044573+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-16T10:11:44.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-03-16T10:11:44.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.028131+0000', 'last_change': '2024-03-16T10:11:40.028131+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.028131+0000', 'last_undegraded': '2024-03-16T10:11:40.028131+0000', 'last_fullsized': '2024-03-16T10:11:40.028131+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': ['3', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-16T10:11:44.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-03-16T10:11:44.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.040072+0000', 'last_change': '2024-03-16T10:11:40.040072+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.040072+0000', 'last_undegraded': '2024-03-16T10:11:40.040072+0000', 'last_fullsized': '2024-03-16T10:11:40.040072+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-16T10:11:44.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-03-16T10:11:44.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.045399+0000', 'last_change': '2024-03-16T10:11:40.038809+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.045399+0000', 'last_undegraded': '2024-03-16T10:11:40.045399+0000', 'last_fullsized': '2024-03-16T10:11:40.045399+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-16T10:11:44.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-03-16T10:11:44.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.049110+0000', 'last_change': '2024-03-16T10:11:40.049110+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.049110+0000', 'last_undegraded': '2024-03-16T10:11:40.049110+0000', 'last_fullsized': '2024-03-16T10:11:40.049110+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': ['7', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-16T10:11:44.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-03-16T10:11:44.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.043630+0000', 'last_change': '2024-03-16T10:11:40.043630+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.043630+0000', 'last_undegraded': '2024-03-16T10:11:40.043630+0000', 'last_fullsized': '2024-03-16T10:11:40.043630+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-16T10:11:44.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-03-16T10:11:44.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.035844+0000', 'last_change': '2024-03-16T10:11:40.035844+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.035844+0000', 'last_undegraded': '2024-03-16T10:11:40.035844+0000', 'last_fullsized': '2024-03-16T10:11:40.035844+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': ['5', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-16T10:11:44.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-03-16T10:11:44.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.041855+0000', 'last_change': '2024-03-16T10:11:40.041855+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.041855+0000', 'last_undegraded': '2024-03-16T10:11:40.041855+0000', 'last_fullsized': '2024-03-16T10:11:40.041855+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': ['7', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-16T10:11:44.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-03-16T10:11:44.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.049662+0000', 'last_change': '2024-03-16T10:11:40.049662+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.049662+0000', 'last_undegraded': '2024-03-16T10:11:40.049662+0000', 'last_fullsized': '2024-03-16T10:11:40.049662+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-16T10:11:44.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-03-16T10:11:44.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.023769+0000', 'last_change': '2024-03-16T10:11:40.023769+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.023769+0000', 'last_undegraded': '2024-03-16T10:11:40.023769+0000', 'last_fullsized': '2024-03-16T10:11:40.023769+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': ['2', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-16T10:11:44.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-03-16T10:11:44.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.027173+0000', 'last_change': '2024-03-16T10:11:40.027173+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.027173+0000', 'last_undegraded': '2024-03-16T10:11:40.027173+0000', 'last_fullsized': '2024-03-16T10:11:40.027173+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': ['0', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-16T10:11:44.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-03-16T10:11:44.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.028496+0000', 'last_change': '2024-03-16T10:11:40.028496+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.028496+0000', 'last_undegraded': '2024-03-16T10:11:40.028496+0000', 'last_fullsized': '2024-03-16T10:11:40.028496+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': ['3', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-16T10:11:44.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-03-16T10:11:44.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.042755+0000', 'last_change': '2024-03-16T10:11:40.042755+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.042755+0000', 'last_undegraded': '2024-03-16T10:11:40.042755+0000', 'last_fullsized': '2024-03-16T10:11:40.042755+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': ['5', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-16T10:11:44.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-03-16T10:11:44.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.041848+0000', 'last_change': '2024-03-16T10:11:40.041848+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.041848+0000', 'last_undegraded': '2024-03-16T10:11:40.041848+0000', 'last_fullsized': '2024-03-16T10:11:40.041848+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-16T10:11:44.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-03-16T10:11:44.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.034714+0000', 'last_change': '2024-03-16T10:11:40.028766+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.034714+0000', 'last_undegraded': '2024-03-16T10:11:40.034714+0000', 'last_fullsized': '2024-03-16T10:11:40.034714+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': ['6', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-16T10:11:44.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-03-16T10:11:44.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.039748+0000', 'last_change': '2024-03-16T10:11:40.039748+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.039748+0000', 'last_undegraded': '2024-03-16T10:11:40.039748+0000', 'last_fullsized': '2024-03-16T10:11:40.039748+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': ['7', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-16T10:11:44.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-03-16T10:11:44.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.027172+0000', 'last_change': '2024-03-16T10:11:40.027172+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.027172+0000', 'last_undegraded': '2024-03-16T10:11:40.027172+0000', 'last_fullsized': '2024-03-16T10:11:40.027172+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': ['0', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-16T10:11:44.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-03-16T10:11:44.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.041163+0000', 'last_change': '2024-03-16T10:11:40.041163+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.041163+0000', 'last_undegraded': '2024-03-16T10:11:40.041163+0000', 'last_fullsized': '2024-03-16T10:11:40.041163+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': ['7', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-16T10:11:44.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-03-16T10:11:44.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.040415+0000', 'last_change': '2024-03-16T10:11:40.040415+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.040415+0000', 'last_undegraded': '2024-03-16T10:11:40.040415+0000', 'last_fullsized': '2024-03-16T10:11:40.040415+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': ['5', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-16T10:11:44.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-03-16T10:11:44.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 6, 'reported_epoch': 29, 'state': 'active', 'last_fresh': '2024-03-16T10:11:40.070794+0000', 'last_change': '2024-03-16T10:11:40.070642+0000', 'last_active': '2024-03-16T10:11:40.070794+0000', 'last_peered': '2024-03-16T10:11:40.070794+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '2024-03-16T10:11:40.070642+0000', 'last_became_peered': '2024-03-16T10:11:40.070642+0000', 'last_unstale': '2024-03-16T10:11:40.070794+0000', 'last_undegraded': '2024-03-16T10:11:40.070794+0000', 'last_fullsized': '2024-03-16T10:11:40.070794+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': ['4', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-16T10:11:44.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-03-16T10:11:44.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.029382+0000', 'last_change': '2024-03-16T10:11:40.029382+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.029382+0000', 'last_undegraded': '2024-03-16T10:11:40.029382+0000', 'last_fullsized': '2024-03-16T10:11:40.029382+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': ['1', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-16T10:11:44.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-03-16T10:11:44.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.046622+0000', 'last_change': '2024-03-16T10:11:40.046622+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.046622+0000', 'last_undegraded': '2024-03-16T10:11:40.046622+0000', 'last_fullsized': '2024-03-16T10:11:40.046622+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-16T10:11:44.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-03-16T10:11:44.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.045474+0000', 'last_change': '2024-03-16T10:11:40.045474+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.045474+0000', 'last_undegraded': '2024-03-16T10:11:40.045474+0000', 'last_fullsized': '2024-03-16T10:11:40.045474+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-16T10:11:44.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-03-16T10:11:44.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.029679+0000', 'last_change': '2024-03-16T10:11:40.029679+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.029679+0000', 'last_undegraded': '2024-03-16T10:11:40.029679+0000', 'last_fullsized': '2024-03-16T10:11:40.029679+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': ['1', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-16T10:11:44.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-03-16T10:11:44.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.040610+0000', 'last_change': '2024-03-16T10:11:40.040610+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.040610+0000', 'last_undegraded': '2024-03-16T10:11:40.040610+0000', 'last_fullsized': '2024-03-16T10:11:40.040610+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': ['7', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-16T10:11:44.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-03-16T10:11:44.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 6, 'reported_epoch': 29, 'state': 'active', 'last_fresh': '2024-03-16T10:11:40.058574+0000', 'last_change': '2024-03-16T10:11:40.058392+0000', 'last_active': '2024-03-16T10:11:40.058574+0000', 'last_peered': '2024-03-16T10:11:40.058574+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '2024-03-16T10:11:40.058392+0000', 'last_became_peered': '2024-03-16T10:11:40.058392+0000', 'last_unstale': '2024-03-16T10:11:40.058574+0000', 'last_undegraded': '2024-03-16T10:11:40.058574+0000', 'last_fullsized': '2024-03-16T10:11:40.058574+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': ['6', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-16T10:11:44.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-03-16T10:11:44.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.037470+0000', 'last_change': '2024-03-16T10:11:40.037470+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.037470+0000', 'last_undegraded': '2024-03-16T10:11:40.037470+0000', 'last_fullsized': '2024-03-16T10:11:40.037470+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': ['5', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-16T10:11:44.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-03-16T10:11:44.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.031547+0000', 'last_change': '2024-03-16T10:11:40.031547+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.031547+0000', 'last_undegraded': '2024-03-16T10:11:40.031547+0000', 'last_fullsized': '2024-03-16T10:11:40.031547+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': ['1', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-16T10:11:44.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-03-16T10:11:44.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.024225+0000', 'last_change': '2024-03-16T10:11:40.024225+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.024225+0000', 'last_undegraded': '2024-03-16T10:11:40.024225+0000', 'last_fullsized': '2024-03-16T10:11:40.024225+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': ['2', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-16T10:11:44.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-03-16T10:11:44.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.039125+0000', 'last_change': '2024-03-16T10:11:40.039125+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.039125+0000', 'last_undegraded': '2024-03-16T10:11:40.039125+0000', 'last_fullsized': '2024-03-16T10:11:40.039125+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': ['7', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-16T10:11:44.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-03-16T10:11:44.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.024301+0000', 'last_change': '2024-03-16T10:11:40.024301+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.024301+0000', 'last_undegraded': '2024-03-16T10:11:40.024301+0000', 'last_fullsized': '2024-03-16T10:11:40.024301+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': ['2', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-16T10:11:44.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-03-16T10:11:44.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.036465+0000', 'last_change': '2024-03-16T10:11:40.036465+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.036465+0000', 'last_undegraded': '2024-03-16T10:11:40.036465+0000', 'last_fullsized': '2024-03-16T10:11:40.036465+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-16T10:11:44.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-03-16T10:11:44.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.037552+0000', 'last_change': '2024-03-16T10:11:40.031811+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.037552+0000', 'last_undegraded': '2024-03-16T10:11:40.037552+0000', 'last_fullsized': '2024-03-16T10:11:40.037552+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': ['6', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-16T10:11:44.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-03-16T10:11:44.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.027157+0000', 'last_change': '2024-03-16T10:11:40.027157+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.027157+0000', 'last_undegraded': '2024-03-16T10:11:40.027157+0000', 'last_fullsized': '2024-03-16T10:11:40.027157+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': ['0', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-16T10:11:44.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-03-16T10:11:44.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 4, 'reported_epoch': 29, 'state': 'creating+activating', 'last_fresh': '2024-03-16T10:11:40.045609+0000', 'last_change': '2024-03-16T10:11:40.039231+0000', 'last_active': '2024-03-16T10:11:39.007282+0000', 'last_peered': '2024-03-16T10:11:39.007282+0000', 'last_clean': '2024-03-16T10:11:39.007282+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-16T10:11:40.045609+0000', 'last_undegraded': '2024-03-16T10:11:40.045609+0000', 'last_fullsized': '2024-03-16T10:11:40.045609+0000', 'mapping_epoch': 28, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 28, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'last_clean_scrub_stamp': '2024-03-16T10:11:39.007282+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-16T10:11:44.030 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-16T10:11:44.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.172+0000 7f47d2339640 1 Processor -- start 2024-03-16T10:11:44.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.172+0000 7f47d2339640 1 -- start start 2024-03-16T10:11:44.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.172+0000 7f47d2339640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47cc09fa30 0x7f47cc1482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:44.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.172+0000 7f47d2339640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47cc06b8d0 con 0x7f47cc072ab0 2024-03-16T10:11:44.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.172+0000 7f47d2339640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47cc0b4f60 con 0x7f47cc06b4f0 2024-03-16T10:11:44.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.172+0000 7f47d2339640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f47cc0b4d10 con 0x7f47cc09fa30 2024-03-16T10:11:44.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47d08d6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47cc09fa30 0x7f47cc1482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:44.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47d08d6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f47cc09fa30 0x7f47cc1482d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52514/0 (socket says 172.21.15.17:52514) 2024-03-16T10:11:44.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47d08d6640 1 -- 172.21.15.17:0/1841437928 learned_addr learned my addr 172.21.15.17:0/1841437928 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:44.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47d08d6640 1 -- 172.21.15.17:0/1841437928 >> v1:172.21.15.174:6789/0 conn(0x7f47cc06b4f0 legacy=0x7f47cc09f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47d08d6640 1 -- 172.21.15.17:0/1841437928 >> v1:172.21.15.17:6789/0 conn(0x7f47cc072ab0 legacy=0x7f47cc147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47caffd640 1 -- 172.21.15.17:0/1841437928 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 489994409 0 0) 0x7f47cc06b8d0 con 0x7f47cc072ab0 2024-03-16T10:11:44.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47d08d6640 1 -- 172.21.15.17:0/1841437928 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc0b4530 con 0x7f47cc09fa30 2024-03-16T10:11:44.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47d08d6640 1 --2- 172.21.15.17:0/1841437928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47cc09fa30 0x7f47cc1482d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009e50 tx=0x7f47c401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4ea2177fae48a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:44.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47caffd640 1 -- 172.21.15.17:0/1841437928 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c40a56e0 con 0x7f47cc09fa30 2024-03-16T10:11:44.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47caffd640 1 -- 172.21.15.17:0/1841437928 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47c40a5880 con 0x7f47cc09fa30 2024-03-16T10:11:44.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47caffd640 1 -- 172.21.15.17:0/1841437928 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c40a5c50 con 0x7f47cc09fa30 2024-03-16T10:11:44.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47d2339640 1 -- 172.21.15.17:0/1841437928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47cc09fa30 msgr2=0x7f47cc1482d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47d2339640 1 --2- 172.21.15.17:0/1841437928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47cc09fa30 0x7f47cc1482d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009e50 tx=0x7f47c401c910 comp rx=0 tx=0).stop 2024-03-16T10:11:44.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47d2339640 1 -- 172.21.15.17:0/1841437928 shutdown_connections 2024-03-16T10:11:44.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47d2339640 1 --2- 172.21.15.17:0/1841437928 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47cc09fa30 0x7f47cc1482d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47d2339640 1 -- 172.21.15.17:0/1841437928 >> 172.21.15.17:0/1841437928 conn(0x7f47cc0b9a30 msgr2=0x7f47cc0b9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:44.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47d2339640 1 -- 172.21.15.17:0/1841437928 shutdown_connections 2024-03-16T10:11:44.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.176+0000 7f47d2339640 1 -- 172.21.15.17:0/1841437928 wait complete. 2024-03-16T10:11:44.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47d2339640 1 Processor -- start 2024-03-16T10:11:44.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47d2339640 1 -- start start 2024-03-16T10:11:44.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47d2339640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47cc06b4f0 0x7f47cc1dff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:44.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47d2339640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47cc072ab0 0x7f47cc139ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:44.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47d2339640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47cc09fa30 0x7f47cc1e34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:44.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47d2339640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc155e10 con 0x7f47cc072ab0 2024-03-16T10:11:44.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47d2339640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47cc0b52d0 con 0x7f47cc06b4f0 2024-03-16T10:11:44.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47d2339640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f47cc155f90 con 0x7f47cc09fa30 2024-03-16T10:11:44.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47cb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47cc072ab0 0x7f47cc139ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:44.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47cb7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47cc072ab0 0x7f47cc139ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52518/0 (socket says 172.21.15.17:52518) 2024-03-16T10:11:44.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47cb7fe640 1 -- 172.21.15.17:0/3434593317 learned_addr learned my addr 172.21.15.17:0/3434593317 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:44.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47cbfff640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47cc06b4f0 0x7f47cc1dff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:44.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47d08d6640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47cc09fa30 0x7f47cc1e34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:44.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47cb7fe640 1 -- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47cc09fa30 msgr2=0x7f47cc1e34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47cb7fe640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47cc09fa30 0x7f47cc1e34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47cb7fe640 1 -- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47cc06b4f0 msgr2=0x7f47cc1dff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47cb7fe640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47cc06b4f0 0x7f47cc1dff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47cb7fe640 1 -- 172.21.15.17:0/3434593317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47cc141010 con 0x7f47cc072ab0 2024-03-16T10:11:44.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47d08d6640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47cc09fa30 0x7f47cc1e34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:44.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47cb7fe640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47cc072ab0 0x7f47cc139ee0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c3e0 tx=0x7f47bc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:44.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47c8ff9640 1 -- 172.21.15.17:0/3434593317 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc017070 con 0x7f47cc072ab0 2024-03-16T10:11:44.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47c8ff9640 1 -- 172.21.15.17:0/3434593317 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47bc003600 con 0x7f47cc072ab0 2024-03-16T10:11:44.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47d2339640 1 -- 172.21.15.17:0/3434593317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47cc13fd70 con 0x7f47cc072ab0 2024-03-16T10:11:44.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47c8ff9640 1 -- 172.21.15.17:0/3434593317 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0039f0 con 0x7f47cc072ab0 2024-03-16T10:11:44.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47d2339640 1 -- 172.21.15.17:0/3434593317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47cc1e04b0 con 0x7f47cc072ab0 2024-03-16T10:11:44.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47d2339640 1 -- 172.21.15.17:0/3434593317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47cc140160 con 0x7f47cc072ab0 2024-03-16T10:11:44.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47c8ff9640 1 -- 172.21.15.17:0/3434593317 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f47bc01a070 con 0x7f47cc072ab0 2024-03-16T10:11:44.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47c8ff9640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f47b8067660 0x7f47b8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:44.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47c8ff9640 1 -- 172.21.15.17:0/3434593317 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f47bc084ba0 con 0x7f47cc072ab0 2024-03-16T10:11:44.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.180+0000 7f47cbfff640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f47b8067660 0x7f47b8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:44.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.184+0000 7f47cbfff640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f47b8067660 0x7f47b8069b20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f47c0000f70 tx=0x7f47c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:44.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.184+0000 7f47c8ff9640 1 -- 172.21.15.17:0/3434593317 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f47cc141010 con 0x7f47cc072ab0 2024-03-16T10:11:44.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.396+0000 7f47d2339640 1 -- 172.21.15.17:0/3434593317 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f47cc13ff40 con 0x7f47cc072ab0 2024-03-16T10:11:44.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.396+0000 7f47c8ff9640 1 -- 172.21.15.17:0/3434593317 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2489 (secure 0 0 0) 0x7f47bc0558b0 con 0x7f47cc072ab0 2024-03-16T10:11:44.402 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:44.402 INFO:teuthology.orchestra.run.smithi017.stdout:{"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":71,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":30,"num_osds":8,"num_up_osds":8,"osd_up_since":1710583844,"num_in_osds":8,"osd_in_since":1710583835,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":61},{"state_name":"creating+activating","count":34},{"state_name":"active","count":2}],"num_pgs":97,"num_pools":3,"num_objects":21,"data_bytes":591804,"bytes_used":242089984,"bytes_avail":772852023296,"bytes_total":773094113280,"inactive_pgs_ratio":0.3505154550075531,"write_bytes_sec":2047,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":15,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4953}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-03-16T10:11:40.028829+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-16T10:11:44.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.400+0000 7f47d2339640 1 -- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f47b8067660 msgr2=0x7f47b8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.400+0000 7f47d2339640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f47b8067660 0x7f47b8069b20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f47c0000f70 tx=0x7f47c0009040 comp rx=0 tx=0).stop 2024-03-16T10:11:44.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.400+0000 7f47d2339640 1 -- 172.21.15.17:0/3434593317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47cc072ab0 msgr2=0x7f47cc139ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.400+0000 7f47d2339640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47cc072ab0 0x7f47cc139ee0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c3e0 tx=0x7f47bc00c8a0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.400+0000 7f47d2339640 1 -- 172.21.15.17:0/3434593317 shutdown_connections 2024-03-16T10:11:44.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.400+0000 7f47d2339640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f47b8067660 0x7f47b8069b20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.400+0000 7f47d2339640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f47cc09fa30 0x7f47cc1e34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.400+0000 7f47d2339640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47cc072ab0 0x7f47cc139ee0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.400+0000 7f47d2339640 1 --2- 172.21.15.17:0/3434593317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47cc06b4f0 0x7f47cc1dff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.400+0000 7f47d2339640 1 -- 172.21.15.17:0/3434593317 >> 172.21.15.17:0/3434593317 conn(0x7f47cc0b9a30 msgr2=0x7f47cc0b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:44.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.400+0000 7f47d2339640 1 -- 172.21.15.17:0/3434593317 shutdown_connections 2024-03-16T10:11:44.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.400+0000 7f47d2339640 1 -- 172.21.15.17:0/3434593317 wait complete. 2024-03-16T10:11:44.419 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-16T10:11:44.419 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:11:44.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.524+0000 7fc1de5e4640 1 Processor -- start 2024-03-16T10:11:44.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.524+0000 7fc1de5e4640 1 -- start start 2024-03-16T10:11:44.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.524+0000 7fc1de5e4640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1d8141940 0x7fc1d8152df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:44.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.524+0000 7fc1de5e4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d814b0f0 con 0x7fc1d806d040 2024-03-16T10:11:44.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.524+0000 7fc1de5e4640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1d806f610 con 0x7fc1d814ad10 2024-03-16T10:11:44.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.524+0000 7fc1de5e4640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc1d806c960 con 0x7fc1d8141940 2024-03-16T10:11:44.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.528+0000 7fc1dcb81640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1d8141940 0x7fc1d8152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:44.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.528+0000 7fc1dcb81640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc1d8141940 0x7fc1d8152df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52534/0 (socket says 172.21.15.17:52534) 2024-03-16T10:11:44.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.528+0000 7fc1d77fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fc1d806d040 0x7fc1d8141230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48508/0 (socket says 172.21.15.17:48508) 2024-03-16T10:11:44.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.528+0000 7fc1dcb81640 1 -- 172.21.15.17:0/1736518804 learned_addr learned my addr 172.21.15.17:0/1736518804 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:44.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.528+0000 7fc1dcb81640 1 -- 172.21.15.17:0/1736518804 >> v1:172.21.15.174:6789/0 conn(0x7fc1d814ad10 legacy=0x7fc1d814b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.528+0000 7fc1dcb81640 1 -- 172.21.15.17:0/1736518804 >> v1:172.21.15.17:6789/0 conn(0x7fc1d806d040 legacy=0x7fc1d8141230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.528+0000 7fc1d6ffd640 1 -- 172.21.15.17:0/1736518804 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1516359594 0 0) 0x7fc1d814b0f0 con 0x7fc1d806d040 2024-03-16T10:11:44.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.528+0000 7fc1dcb81640 1 -- 172.21.15.17:0/1736518804 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d80bbbf0 con 0x7fc1d8141940 2024-03-16T10:11:44.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.528+0000 7fc1dcb81640 1 --2- 172.21.15.17:0/1736518804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d8141940 0x7fc1d8152df0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009e50 tx=0x7fc1d001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138405bf331afd87 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:44.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.528+0000 7fc1d6ffd640 1 -- 172.21.15.17:0/1736518804 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00a5700 con 0x7fc1d8141940 2024-03-16T10:11:44.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.528+0000 7fc1d6ffd640 1 -- 172.21.15.17:0/1736518804 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d00a58a0 con 0x7fc1d8141940 2024-03-16T10:11:44.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.528+0000 7fc1d6ffd640 1 -- 172.21.15.17:0/1736518804 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00a5c70 con 0x7fc1d8141940 2024-03-16T10:11:44.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 -- 172.21.15.17:0/1736518804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d8141940 msgr2=0x7fc1d8152df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 --2- 172.21.15.17:0/1736518804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d8141940 0x7fc1d8152df0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009e50 tx=0x7fc1d001ca30 comp rx=0 tx=0).stop 2024-03-16T10:11:44.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 -- 172.21.15.17:0/1736518804 shutdown_connections 2024-03-16T10:11:44.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 --2- 172.21.15.17:0/1736518804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d8141940 0x7fc1d8152df0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 -- 172.21.15.17:0/1736518804 >> 172.21.15.17:0/1736518804 conn(0x7fc1d80718a0 msgr2=0x7fc1d8071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:44.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 -- 172.21.15.17:0/1736518804 shutdown_connections 2024-03-16T10:11:44.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 -- 172.21.15.17:0/1736518804 wait complete. 2024-03-16T10:11:44.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 Processor -- start 2024-03-16T10:11:44.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 -- start start 2024-03-16T10:11:44.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1d806d040 0x7fc1d8147de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:44.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc1d8141940 0x7fc1d8142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:44.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d814ad10 0x7fc1d81427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:44.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d8155d60 con 0x7fc1d814ad10 2024-03-16T10:11:44.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d806fcb0 con 0x7fc1d806d040 2024-03-16T10:11:44.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1de5e4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d8155ee0 con 0x7fc1d8141940 2024-03-16T10:11:44.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1dcb81640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d814ad10 0x7fc1d81427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:44.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1dcb81640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d814ad10 0x7fc1d81427c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52544/0 (socket says 172.21.15.17:52544) 2024-03-16T10:11:44.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.532+0000 7fc1dcb81640 1 -- 172.21.15.17:0/3899141488 learned_addr learned my addr 172.21.15.17:0/3899141488 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:44.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1d77fe640 1 --2- 172.21.15.17:0/3899141488 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc1d8141940 0x7fc1d8142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:44.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1d7fff640 1 --2- 172.21.15.17:0/3899141488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1d806d040 0x7fc1d8147de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:44.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1dcb81640 1 -- 172.21.15.17:0/3899141488 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc1d8141940 msgr2=0x7fc1d8142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1dcb81640 1 --2- 172.21.15.17:0/3899141488 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc1d8141940 0x7fc1d8142280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1dcb81640 1 -- 172.21.15.17:0/3899141488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1d806d040 msgr2=0x7fc1d8147de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1dcb81640 1 --2- 172.21.15.17:0/3899141488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1d806d040 0x7fc1d8147de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1dcb81640 1 -- 172.21.15.17:0/3899141488 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d8144300 con 0x7fc1d814ad10 2024-03-16T10:11:44.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1dcb81640 1 --2- 172.21.15.17:0/3899141488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d814ad10 0x7fc1d81427c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009e50 tx=0x7fc1d00197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:44.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1d4ff9640 1 -- 172.21.15.17:0/3899141488 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00bc030 con 0x7fc1d814ad10 2024-03-16T10:11:44.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1d4ff9640 1 -- 172.21.15.17:0/3899141488 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d00b8470 con 0x7fc1d814ad10 2024-03-16T10:11:44.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1de5e4640 1 -- 172.21.15.17:0/3899141488 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d8144d00 con 0x7fc1d814ad10 2024-03-16T10:11:44.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1d4ff9640 1 -- 172.21.15.17:0/3899141488 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00b8820 con 0x7fc1d814ad10 2024-03-16T10:11:44.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1de5e4640 1 -- 172.21.15.17:0/3899141488 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d8144020 con 0x7fc1d814ad10 2024-03-16T10:11:44.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1de5e4640 1 -- 172.21.15.17:0/3899141488 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d806a130 con 0x7fc1d814ad10 2024-03-16T10:11:44.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1d4ff9640 1 -- 172.21.15.17:0/3899141488 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fc1d00bf070 con 0x7fc1d814ad10 2024-03-16T10:11:44.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1d4ff9640 1 --2- 172.21.15.17:0/3899141488 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc1c4067660 0x7fc1c4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:44.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1d4ff9640 1 -- 172.21.15.17:0/3899141488 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fc1d0120c40 con 0x7fc1d814ad10 2024-03-16T10:11:44.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.536+0000 7fc1d7fff640 1 --2- 172.21.15.17:0/3899141488 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc1c4067660 0x7fc1c4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:44.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.540+0000 7fc1d7fff640 1 --2- 172.21.15.17:0/3899141488 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc1c4067660 0x7fc1c4069b20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8000f70 tx=0x7fc1c8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:44.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.540+0000 7fc1d4ff9640 1 -- 172.21.15.17:0/3899141488 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc1d8144d00 con 0x7fc1d814ad10 2024-03-16T10:11:44.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.688+0000 7fc1de5e4640 1 -- 172.21.15.17:0/3899141488 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1d806c5f0 con 0x7fc1c4067660 2024-03-16T10:11:44.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.708+0000 7fc1d4ff9640 1 -- 172.21.15.17:0/3899141488 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249410 (secure 0 0 0) 0x7fc1d806c5f0 con 0x7fc1c4067660 2024-03-16T10:11:44.715 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:44.717 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-16T10:11:44.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.712+0000 7fc1de5e4640 1 -- 172.21.15.17:0/3899141488 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc1c4067660 msgr2=0x7fc1c4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.712+0000 7fc1de5e4640 1 --2- 172.21.15.17:0/3899141488 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc1c4067660 0x7fc1c4069b20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8000f70 tx=0x7fc1c8009040 comp rx=0 tx=0).stop 2024-03-16T10:11:44.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.712+0000 7fc1de5e4640 1 -- 172.21.15.17:0/3899141488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d814ad10 msgr2=0x7fc1d81427c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:44.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.712+0000 7fc1de5e4640 1 --2- 172.21.15.17:0/3899141488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d814ad10 0x7fc1d81427c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0009e50 tx=0x7fc1d00197d0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.712+0000 7fc1de5e4640 1 -- 172.21.15.17:0/3899141488 shutdown_connections 2024-03-16T10:11:44.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.712+0000 7fc1de5e4640 1 --2- 172.21.15.17:0/3899141488 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc1c4067660 0x7fc1c4069b20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.712+0000 7fc1de5e4640 1 --2- 172.21.15.17:0/3899141488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1d814ad10 0x7fc1d81427c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.712+0000 7fc1de5e4640 1 --2- 172.21.15.17:0/3899141488 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc1d8141940 0x7fc1d8142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.712+0000 7fc1de5e4640 1 --2- 172.21.15.17:0/3899141488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1d806d040 0x7fc1d8147de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:44.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.712+0000 7fc1de5e4640 1 -- 172.21.15.17:0/3899141488 >> 172.21.15.17:0/3899141488 conn(0x7fc1d80718a0 msgr2=0x7fc1d80b5ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:44.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.712+0000 7fc1de5e4640 1 -- 172.21.15.17:0/3899141488 shutdown_connections 2024-03-16T10:11:44.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:44.712+0000 7fc1de5e4640 1 -- 172.21.15.17:0/3899141488 wait complete. 2024-03-16T10:11:44.732 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-03-16T10:11:44.030116+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591804,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":201,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236416,"kb_used_data":3592,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738304,"statfs":{"total":773094113280,"available":772852023296,"internally_reserved":0,"allocated":3678208,"data_stored":2105600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1436,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001970"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.029409+0000","last_change":"2024-03-16T10:11:40.029409+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.029409+0000","last_undegraded":"2024-03-16T10:11:40.029409+0000","last_fullsized":"2024-03-16T10:11:40.029409+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046589+0000","last_change":"2024-03-16T10:11:40.042825+0000","last_active":"2024-03-16T10:11:40.046589+0000","last_peered":"2024-03-16T10:11:40.046589+0000","last_clean":"2024-03-16T10:11:40.046589+0000","last_became_active":"2024-03-16T10:11:40.042533+0000","last_became_peered":"2024-03-16T10:11:40.042533+0000","last_unstale":"2024-03-16T10:11:40.046589+0000","last_undegraded":"2024-03-16T10:11:40.046589+0000","last_fullsized":"2024-03-16T10:11:40.046589+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:09:41.244065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.044573+0000","last_change":"2024-03-16T10:11:40.044573+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.044573+0000","last_undegraded":"2024-03-16T10:11:40.044573+0000","last_fullsized":"2024-03-16T10:11:40.044573+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048335+0000","last_change":"2024-03-16T10:11:40.045382+0000","last_active":"2024-03-16T10:11:40.048335+0000","last_peered":"2024-03-16T10:11:40.048335+0000","last_clean":"2024-03-16T10:11:40.048335+0000","last_became_active":"2024-03-16T10:11:40.045183+0000","last_became_peered":"2024-03-16T10:11:40.045183+0000","last_unstale":"2024-03-16T10:11:40.048335+0000","last_undegraded":"2024-03-16T10:11:40.048335+0000","last_fullsized":"2024-03-16T10:11:40.048335+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:04.144054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045289+0000","last_change":"2024-03-16T10:11:40.044298+0000","last_active":"2024-03-16T10:11:40.045289+0000","last_peered":"2024-03-16T10:11:40.045289+0000","last_clean":"2024-03-16T10:11:40.045289+0000","last_became_active":"2024-03-16T10:11:40.043984+0000","last_became_peered":"2024-03-16T10:11:40.043984+0000","last_unstale":"2024-03-16T10:11:40.045289+0000","last_undegraded":"2024-03-16T10:11:40.045289+0000","last_fullsized":"2024-03-16T10:11:40.045289+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:50.672600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.028131+0000","last_change":"2024-03-16T10:11:40.028131+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.028131+0000","last_undegraded":"2024-03-16T10:11:40.028131+0000","last_fullsized":"2024-03-16T10:11:40.028131+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":["3","5"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.040072+0000","last_change":"2024-03-16T10:11:40.040072+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.040072+0000","last_undegraded":"2024-03-16T10:11:40.040072+0000","last_fullsized":"2024-03-16T10:11:40.040072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.045399+0000","last_change":"2024-03-16T10:11:40.038809+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.045399+0000","last_undegraded":"2024-03-16T10:11:40.045399+0000","last_fullsized":"2024-03-16T10:11:40.045399+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.047641+0000","last_change":"2024-03-16T10:11:40.047641+0000","last_active":"2024-03-16T10:11:40.047641+0000","last_peered":"2024-03-16T10:11:40.047641+0000","last_clean":"2024-03-16T10:11:40.047641+0000","last_became_active":"2024-03-16T10:11:40.046666+0000","last_became_peered":"2024-03-16T10:11:40.046666+0000","last_unstale":"2024-03-16T10:11:40.047641+0000","last_undegraded":"2024-03-16T10:11:40.047641+0000","last_fullsized":"2024-03-16T10:11:40.047641+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:02.965529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048493+0000","last_change":"2024-03-16T10:11:40.047611+0000","last_active":"2024-03-16T10:11:40.048493+0000","last_peered":"2024-03-16T10:11:40.048493+0000","last_clean":"2024-03-16T10:11:40.048493+0000","last_became_active":"2024-03-16T10:11:40.047371+0000","last_became_peered":"2024-03-16T10:11:40.047371+0000","last_unstale":"2024-03-16T10:11:40.048493+0000","last_undegraded":"2024-03-16T10:11:40.048493+0000","last_fullsized":"2024-03-16T10:11:40.048493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:34:24.572872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.049110+0000","last_change":"2024-03-16T10:11:40.049110+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.049110+0000","last_undegraded":"2024-03-16T10:11:40.049110+0000","last_fullsized":"2024-03-16T10:11:40.049110+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.043630+0000","last_change":"2024-03-16T10:11:40.043630+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.043630+0000","last_undegraded":"2024-03-16T10:11:40.043630+0000","last_fullsized":"2024-03-16T10:11:40.043630+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.039941+0000","last_change":"2024-03-16T10:11:40.038996+0000","last_active":"2024-03-16T10:11:40.039941+0000","last_peered":"2024-03-16T10:11:40.039941+0000","last_clean":"2024-03-16T10:11:40.039941+0000","last_became_active":"2024-03-16T10:11:40.036020+0000","last_became_peered":"2024-03-16T10:11:40.036020+0000","last_unstale":"2024-03-16T10:11:40.039941+0000","last_undegraded":"2024-03-16T10:11:40.039941+0000","last_fullsized":"2024-03-16T10:11:40.039941+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:28:23.989331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046436+0000","last_change":"2024-03-16T10:11:40.042699+0000","last_active":"2024-03-16T10:11:40.046436+0000","last_peered":"2024-03-16T10:11:40.046436+0000","last_clean":"2024-03-16T10:11:40.046436+0000","last_became_active":"2024-03-16T10:11:40.042299+0000","last_became_peered":"2024-03-16T10:11:40.042299+0000","last_unstale":"2024-03-16T10:11:40.046436+0000","last_undegraded":"2024-03-16T10:11:40.046436+0000","last_fullsized":"2024-03-16T10:11:40.046436+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:39:53.674994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.035844+0000","last_change":"2024-03-16T10:11:40.035844+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.035844+0000","last_undegraded":"2024-03-16T10:11:40.035844+0000","last_fullsized":"2024-03-16T10:11:40.035844+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.041855+0000","last_change":"2024-03-16T10:11:40.041855+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.041855+0000","last_undegraded":"2024-03-16T10:11:40.041855+0000","last_fullsized":"2024-03-16T10:11:40.041855+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.049662+0000","last_change":"2024-03-16T10:11:40.049662+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.049662+0000","last_undegraded":"2024-03-16T10:11:40.049662+0000","last_fullsized":"2024-03-16T10:11:40.049662+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049450+0000","last_change":"2024-03-16T10:11:40.046723+0000","last_active":"2024-03-16T10:11:40.049450+0000","last_peered":"2024-03-16T10:11:40.049450+0000","last_clean":"2024-03-16T10:11:40.049450+0000","last_became_active":"2024-03-16T10:11:40.043586+0000","last_became_peered":"2024-03-16T10:11:40.043586+0000","last_unstale":"2024-03-16T10:11:40.049450+0000","last_undegraded":"2024-03-16T10:11:40.049450+0000","last_fullsized":"2024-03-16T10:11:40.049450+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:06.536564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046506+0000","last_change":"2024-03-16T10:11:40.043055+0000","last_active":"2024-03-16T10:11:40.046506+0000","last_peered":"2024-03-16T10:11:40.046506+0000","last_clean":"2024-03-16T10:11:40.046506+0000","last_became_active":"2024-03-16T10:11:40.041082+0000","last_became_peered":"2024-03-16T10:11:40.041082+0000","last_unstale":"2024-03-16T10:11:40.046506+0000","last_undegraded":"2024-03-16T10:11:40.046506+0000","last_fullsized":"2024-03-16T10:11:40.046506+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:35:29.488404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.023769+0000","last_change":"2024-03-16T10:11:40.023769+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.023769+0000","last_undegraded":"2024-03-16T10:11:40.023769+0000","last_fullsized":"2024-03-16T10:11:40.023769+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046160+0000","last_change":"2024-03-16T10:11:40.046160+0000","last_active":"2024-03-16T10:11:40.046160+0000","last_peered":"2024-03-16T10:11:40.046160+0000","last_clean":"2024-03-16T10:11:40.046160+0000","last_became_active":"2024-03-16T10:11:40.045457+0000","last_became_peered":"2024-03-16T10:11:40.045457+0000","last_unstale":"2024-03-16T10:11:40.046160+0000","last_undegraded":"2024-03-16T10:11:40.046160+0000","last_fullsized":"2024-03-16T10:11:40.046160+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:58.221102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.050020+0000","last_change":"2024-03-16T10:11:40.050020+0000","last_active":"2024-03-16T10:11:40.050020+0000","last_peered":"2024-03-16T10:11:40.050020+0000","last_clean":"2024-03-16T10:11:40.050020+0000","last_became_active":"2024-03-16T10:11:40.047232+0000","last_became_peered":"2024-03-16T10:11:40.047232+0000","last_unstale":"2024-03-16T10:11:40.050020+0000","last_undegraded":"2024-03-16T10:11:40.050020+0000","last_fullsized":"2024-03-16T10:11:40.050020+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:49.926346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.027173+0000","last_change":"2024-03-16T10:11:40.027173+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.027173+0000","last_undegraded":"2024-03-16T10:11:40.027173+0000","last_fullsized":"2024-03-16T10:11:40.027173+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.050081+0000","last_change":"2024-03-16T10:11:40.041024+0000","last_active":"2024-03-16T10:11:40.050081+0000","last_peered":"2024-03-16T10:11:40.050081+0000","last_clean":"2024-03-16T10:11:40.050081+0000","last_became_active":"2024-03-16T10:11:40.040230+0000","last_became_peered":"2024-03-16T10:11:40.040230+0000","last_unstale":"2024-03-16T10:11:40.050081+0000","last_undegraded":"2024-03-16T10:11:40.050081+0000","last_fullsized":"2024-03-16T10:11:40.050081+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:19.819589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.040254+0000","last_change":"2024-03-16T10:11:40.036983+0000","last_active":"2024-03-16T10:11:40.040254+0000","last_peered":"2024-03-16T10:11:40.040254+0000","last_clean":"2024-03-16T10:11:40.040254+0000","last_became_active":"2024-03-16T10:11:40.036701+0000","last_became_peered":"2024-03-16T10:11:40.036701+0000","last_unstale":"2024-03-16T10:11:40.040254+0000","last_undegraded":"2024-03-16T10:11:40.040254+0000","last_fullsized":"2024-03-16T10:11:40.040254+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:54.938794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.028496+0000","last_change":"2024-03-16T10:11:40.028496+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.028496+0000","last_undegraded":"2024-03-16T10:11:40.028496+0000","last_fullsized":"2024-03-16T10:11:40.028496+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048445+0000","last_change":"2024-03-16T10:11:40.046735+0000","last_active":"2024-03-16T10:11:40.048445+0000","last_peered":"2024-03-16T10:11:40.048445+0000","last_clean":"2024-03-16T10:11:40.048445+0000","last_became_active":"2024-03-16T10:11:40.046202+0000","last_became_peered":"2024-03-16T10:11:40.046202+0000","last_unstale":"2024-03-16T10:11:40.048445+0000","last_undegraded":"2024-03-16T10:11:40.048445+0000","last_fullsized":"2024-03-16T10:11:40.048445+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:48.954207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.042755+0000","last_change":"2024-03-16T10:11:40.042755+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.042755+0000","last_undegraded":"2024-03-16T10:11:40.042755+0000","last_fullsized":"2024-03-16T10:11:40.042755+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.067258+0000","last_change":"2024-03-16T10:11:40.067258+0000","last_active":"2024-03-16T10:11:40.067258+0000","last_peered":"2024-03-16T10:11:40.067258+0000","last_clean":"2024-03-16T10:11:40.067258+0000","last_became_active":"2024-03-16T10:11:40.066423+0000","last_became_peered":"2024-03-16T10:11:40.066423+0000","last_unstale":"2024-03-16T10:11:40.067258+0000","last_undegraded":"2024-03-16T10:11:40.067258+0000","last_fullsized":"2024-03-16T10:11:40.067258+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:59:04.144054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.041848+0000","last_change":"2024-03-16T10:11:40.041848+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.041848+0000","last_undegraded":"2024-03-16T10:11:40.041848+0000","last_fullsized":"2024-03-16T10:11:40.041848+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.034714+0000","last_change":"2024-03-16T10:11:40.028766+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.034714+0000","last_undegraded":"2024-03-16T10:11:40.034714+0000","last_fullsized":"2024-03-16T10:11:40.034714+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":["6","2"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.039748+0000","last_change":"2024-03-16T10:11:40.039748+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.039748+0000","last_undegraded":"2024-03-16T10:11:40.039748+0000","last_fullsized":"2024-03-16T10:11:40.039748+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.027172+0000","last_change":"2024-03-16T10:11:40.027172+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.027172+0000","last_undegraded":"2024-03-16T10:11:40.027172+0000","last_fullsized":"2024-03-16T10:11:40.027172+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"29'4","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030235+0000","last_change":"2024-03-16T10:11:39.061998+0000","last_active":"2024-03-16T10:11:40.030235+0000","last_peered":"2024-03-16T10:11:40.030235+0000","last_clean":"2024-03-16T10:11:40.030235+0000","last_became_active":"2024-03-16T10:11:39.061249+0000","last_became_peered":"2024-03-16T10:11:39.061249+0000","last_unstale":"2024-03-16T10:11:40.030235+0000","last_undegraded":"2024-03-16T10:11:40.030235+0000","last_fullsized":"2024-03-16T10:11:40.030235+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.061676+0000","last_change":"2024-03-16T10:11:40.061676+0000","last_active":"2024-03-16T10:11:40.061676+0000","last_peered":"2024-03-16T10:11:40.061676+0000","last_clean":"2024-03-16T10:11:40.061676+0000","last_became_active":"2024-03-16T10:11:40.060865+0000","last_became_peered":"2024-03-16T10:11:40.060865+0000","last_unstale":"2024-03-16T10:11:40.061676+0000","last_undegraded":"2024-03-16T10:11:40.061676+0000","last_fullsized":"2024-03-16T10:11:40.061676+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:40.792776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027530+0000","last_change":"2024-03-16T10:11:39.048117+0000","last_active":"2024-03-16T10:11:40.027530+0000","last_peered":"2024-03-16T10:11:40.027530+0000","last_clean":"2024-03-16T10:11:40.027530+0000","last_became_active":"2024-03-16T10:11:39.047798+0000","last_became_peered":"2024-03-16T10:11:39.047798+0000","last_unstale":"2024-03-16T10:11:40.027530+0000","last_undegraded":"2024-03-16T10:11:40.027530+0000","last_fullsized":"2024-03-16T10:11:40.027530+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.062870+0000","last_change":"2024-03-16T10:11:40.062870+0000","last_active":"2024-03-16T10:11:40.062870+0000","last_peered":"2024-03-16T10:11:40.062870+0000","last_clean":"2024-03-16T10:11:40.062870+0000","last_became_active":"2024-03-16T10:11:40.054629+0000","last_became_peered":"2024-03-16T10:11:40.054629+0000","last_unstale":"2024-03-16T10:11:40.062870+0000","last_undegraded":"2024-03-16T10:11:40.062870+0000","last_fullsized":"2024-03-16T10:11:40.062870+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:40.792776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"29'2","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049288+0000","last_change":"2024-03-16T10:11:39.038358+0000","last_active":"2024-03-16T10:11:40.049288+0000","last_peered":"2024-03-16T10:11:40.049288+0000","last_clean":"2024-03-16T10:11:40.049288+0000","last_became_active":"2024-03-16T10:11:39.038055+0000","last_became_peered":"2024-03-16T10:11:39.038055+0000","last_unstale":"2024-03-16T10:11:40.049288+0000","last_undegraded":"2024-03-16T10:11:40.049288+0000","last_fullsized":"2024-03-16T10:11:40.049288+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:11:55.739859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.048424+0000","last_change":"2024-03-16T10:11:40.046475+0000","last_active":"2024-03-16T10:11:40.048424+0000","last_peered":"2024-03-16T10:11:40.048424+0000","last_clean":"2024-03-16T10:11:40.048424+0000","last_became_active":"2024-03-16T10:11:40.045876+0000","last_became_peered":"2024-03-16T10:11:40.045876+0000","last_unstale":"2024-03-16T10:11:40.048424+0000","last_undegraded":"2024-03-16T10:11:40.048424+0000","last_fullsized":"2024-03-16T10:11:40.048424+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:35:43.212538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"29'2","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045706+0000","last_change":"2024-03-16T10:11:39.030352+0000","last_active":"2024-03-16T10:11:40.045706+0000","last_peered":"2024-03-16T10:11:40.045706+0000","last_clean":"2024-03-16T10:11:40.045706+0000","last_became_active":"2024-03-16T10:11:39.029031+0000","last_became_peered":"2024-03-16T10:11:39.029031+0000","last_unstale":"2024-03-16T10:11:40.045706+0000","last_undegraded":"2024-03-16T10:11:40.045706+0000","last_fullsized":"2024-03-16T10:11:40.045706+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:07:13.816782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.041163+0000","last_change":"2024-03-16T10:11:40.041163+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.041163+0000","last_undegraded":"2024-03-16T10:11:40.041163+0000","last_fullsized":"2024-03-16T10:11:40.041163+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032683+0000","last_change":"2024-03-16T10:11:39.061267+0000","last_active":"2024-03-16T10:11:40.032683+0000","last_peered":"2024-03-16T10:11:40.032683+0000","last_clean":"2024-03-16T10:11:40.032683+0000","last_became_active":"2024-03-16T10:11:39.060323+0000","last_became_peered":"2024-03-16T10:11:39.060323+0000","last_unstale":"2024-03-16T10:11:40.032683+0000","last_undegraded":"2024-03-16T10:11:40.032683+0000","last_fullsized":"2024-03-16T10:11:40.032683+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:33:59.939254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.040415+0000","last_change":"2024-03-16T10:11:40.040415+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.040415+0000","last_undegraded":"2024-03-16T10:11:40.040415+0000","last_fullsized":"2024-03-16T10:11:40.040415+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027685+0000","last_change":"2024-03-16T10:11:39.061259+0000","last_active":"2024-03-16T10:11:40.027685+0000","last_peered":"2024-03-16T10:11:40.027685+0000","last_clean":"2024-03-16T10:11:40.027685+0000","last_became_active":"2024-03-16T10:11:39.060484+0000","last_became_peered":"2024-03-16T10:11:39.060484+0000","last_unstale":"2024-03-16T10:11:40.027685+0000","last_undegraded":"2024-03-16T10:11:40.027685+0000","last_fullsized":"2024-03-16T10:11:40.027685+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:54.254670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":6,"reported_epoch":29,"state":"active","last_fresh":"2024-03-16T10:11:40.070794+0000","last_change":"2024-03-16T10:11:40.070642+0000","last_active":"2024-03-16T10:11:40.070794+0000","last_peered":"2024-03-16T10:11:40.070794+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"2024-03-16T10:11:40.070642+0000","last_became_peered":"2024-03-16T10:11:40.070642+0000","last_unstale":"2024-03-16T10:11:40.070794+0000","last_undegraded":"2024-03-16T10:11:40.070794+0000","last_fullsized":"2024-03-16T10:11:40.070794+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.034336+0000","last_change":"2024-03-16T10:11:39.033585+0000","last_active":"2024-03-16T10:11:40.034336+0000","last_peered":"2024-03-16T10:11:40.034336+0000","last_clean":"2024-03-16T10:11:40.034336+0000","last_became_active":"2024-03-16T10:11:39.033131+0000","last_became_peered":"2024-03-16T10:11:39.033131+0000","last_unstale":"2024-03-16T10:11:40.034336+0000","last_undegraded":"2024-03-16T10:11:40.034336+0000","last_fullsized":"2024-03-16T10:11:40.034336+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:35:42.208648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.029382+0000","last_change":"2024-03-16T10:11:40.029382+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.029382+0000","last_undegraded":"2024-03-16T10:11:40.029382+0000","last_fullsized":"2024-03-16T10:11:40.029382+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.026886+0000","last_change":"2024-03-16T10:11:39.053374+0000","last_active":"2024-03-16T10:11:40.026886+0000","last_peered":"2024-03-16T10:11:40.026886+0000","last_clean":"2024-03-16T10:11:40.026886+0000","last_became_active":"2024-03-16T10:11:39.051683+0000","last_became_peered":"2024-03-16T10:11:39.051683+0000","last_unstale":"2024-03-16T10:11:40.026886+0000","last_undegraded":"2024-03-16T10:11:40.026886+0000","last_fullsized":"2024-03-16T10:11:40.026886+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:49.668710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.046622+0000","last_change":"2024-03-16T10:11:40.046622+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.046622+0000","last_undegraded":"2024-03-16T10:11:40.046622+0000","last_fullsized":"2024-03-16T10:11:40.046622+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.035769+0000","last_change":"2024-03-16T10:11:39.050941+0000","last_active":"2024-03-16T10:11:40.035769+0000","last_peered":"2024-03-16T10:11:40.035769+0000","last_clean":"2024-03-16T10:11:40.035769+0000","last_became_active":"2024-03-16T10:11:39.050659+0000","last_became_peered":"2024-03-16T10:11:39.050659+0000","last_unstale":"2024-03-16T10:11:40.035769+0000","last_undegraded":"2024-03-16T10:11:40.035769+0000","last_fullsized":"2024-03-16T10:11:40.035769+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:08.229218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046403+0000","last_change":"2024-03-16T10:11:40.043035+0000","last_active":"2024-03-16T10:11:40.046403+0000","last_peered":"2024-03-16T10:11:40.046403+0000","last_clean":"2024-03-16T10:11:40.046403+0000","last_became_active":"2024-03-16T10:11:40.040585+0000","last_became_peered":"2024-03-16T10:11:40.040585+0000","last_unstale":"2024-03-16T10:11:40.046403+0000","last_undegraded":"2024-03-16T10:11:40.046403+0000","last_fullsized":"2024-03-16T10:11:40.046403+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:36:55.327282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027611+0000","last_change":"2024-03-16T10:11:39.045927+0000","last_active":"2024-03-16T10:11:40.027611+0000","last_peered":"2024-03-16T10:11:40.027611+0000","last_clean":"2024-03-16T10:11:40.027611+0000","last_became_active":"2024-03-16T10:11:39.045502+0000","last_became_peered":"2024-03-16T10:11:39.045502+0000","last_unstale":"2024-03-16T10:11:40.027611+0000","last_undegraded":"2024-03-16T10:11:40.027611+0000","last_fullsized":"2024-03-16T10:11:40.027611+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049439+0000","last_change":"2024-03-16T10:11:40.041235+0000","last_active":"2024-03-16T10:11:40.049439+0000","last_peered":"2024-03-16T10:11:40.049439+0000","last_clean":"2024-03-16T10:11:40.049439+0000","last_became_active":"2024-03-16T10:11:40.040856+0000","last_became_peered":"2024-03-16T10:11:40.040856+0000","last_unstale":"2024-03-16T10:11:40.049439+0000","last_undegraded":"2024-03-16T10:11:40.049439+0000","last_fullsized":"2024-03-16T10:11:40.049439+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:38:40.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"29'3","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.046332+0000","last_change":"2024-03-16T10:11:39.052193+0000","last_active":"2024-03-16T10:11:40.046332+0000","last_peered":"2024-03-16T10:11:40.046332+0000","last_clean":"2024-03-16T10:11:40.046332+0000","last_became_active":"2024-03-16T10:11:39.050633+0000","last_became_peered":"2024-03-16T10:11:39.050633+0000","last_unstale":"2024-03-16T10:11:40.046332+0000","last_undegraded":"2024-03-16T10:11:40.046332+0000","last_fullsized":"2024-03-16T10:11:40.046332+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:08.229218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.045474+0000","last_change":"2024-03-16T10:11:40.045474+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.045474+0000","last_undegraded":"2024-03-16T10:11:40.045474+0000","last_fullsized":"2024-03-16T10:11:40.045474+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"29'5","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.053253+0000","last_change":"2024-03-16T10:11:39.042282+0000","last_active":"2024-03-16T10:11:40.053253+0000","last_peered":"2024-03-16T10:11:40.053253+0000","last_clean":"2024-03-16T10:11:40.053253+0000","last_became_active":"2024-03-16T10:11:39.041924+0000","last_became_peered":"2024-03-16T10:11:39.041924+0000","last_unstale":"2024-03-16T10:11:40.053253+0000","last_undegraded":"2024-03-16T10:11:40.053253+0000","last_fullsized":"2024-03-16T10:11:40.053253+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:18.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.029679+0000","last_change":"2024-03-16T10:11:40.029679+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.029679+0000","last_undegraded":"2024-03-16T10:11:40.029679+0000","last_fullsized":"2024-03-16T10:11:40.029679+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.069420+0000","last_change":"2024-03-16T10:11:39.042217+0000","last_active":"2024-03-16T10:11:40.069420+0000","last_peered":"2024-03-16T10:11:40.069420+0000","last_clean":"2024-03-16T10:11:40.069420+0000","last_became_active":"2024-03-16T10:11:39.041514+0000","last_became_peered":"2024-03-16T10:11:39.041514+0000","last_unstale":"2024-03-16T10:11:40.069420+0000","last_undegraded":"2024-03-16T10:11:40.069420+0000","last_fullsized":"2024-03-16T10:11:40.069420+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:12:12.889055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049639+0000","last_change":"2024-03-16T10:11:40.048743+0000","last_active":"2024-03-16T10:11:40.049639+0000","last_peered":"2024-03-16T10:11:40.049639+0000","last_clean":"2024-03-16T10:11:40.049639+0000","last_became_active":"2024-03-16T10:11:40.047008+0000","last_became_peered":"2024-03-16T10:11:40.047008+0000","last_unstale":"2024-03-16T10:11:40.049639+0000","last_undegraded":"2024-03-16T10:11:40.049639+0000","last_fullsized":"2024-03-16T10:11:40.049639+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:56:04.173993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032632+0000","last_change":"2024-03-16T10:11:39.047491+0000","last_active":"2024-03-16T10:11:40.032632+0000","last_peered":"2024-03-16T10:11:40.032632+0000","last_clean":"2024-03-16T10:11:40.032632+0000","last_became_active":"2024-03-16T10:11:39.047085+0000","last_became_peered":"2024-03-16T10:11:39.047085+0000","last_unstale":"2024-03-16T10:11:40.032632+0000","last_undegraded":"2024-03-16T10:11:40.032632+0000","last_fullsized":"2024-03-16T10:11:40.032632+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.040610+0000","last_change":"2024-03-16T10:11:40.040610+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.040610+0000","last_undegraded":"2024-03-16T10:11:40.040610+0000","last_fullsized":"2024-03-16T10:11:40.040610+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.034687+0000","last_change":"2024-03-16T10:11:39.053564+0000","last_active":"2024-03-16T10:11:40.034687+0000","last_peered":"2024-03-16T10:11:40.034687+0000","last_clean":"2024-03-16T10:11:40.034687+0000","last_became_active":"2024-03-16T10:11:39.053183+0000","last_became_peered":"2024-03-16T10:11:39.053183+0000","last_unstale":"2024-03-16T10:11:40.034687+0000","last_undegraded":"2024-03-16T10:11:40.034687+0000","last_fullsized":"2024-03-16T10:11:40.034687+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:45:20.816477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":6,"reported_epoch":29,"state":"active","last_fresh":"2024-03-16T10:11:40.058574+0000","last_change":"2024-03-16T10:11:40.058392+0000","last_active":"2024-03-16T10:11:40.058574+0000","last_peered":"2024-03-16T10:11:40.058574+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"2024-03-16T10:11:40.058392+0000","last_became_peered":"2024-03-16T10:11:40.058392+0000","last_unstale":"2024-03-16T10:11:40.058574+0000","last_undegraded":"2024-03-16T10:11:40.058574+0000","last_fullsized":"2024-03-16T10:11:40.058574+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"29'8","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.053703+0000","last_change":"2024-03-16T10:11:39.035694+0000","last_active":"2024-03-16T10:11:40.053703+0000","last_peered":"2024-03-16T10:11:40.053703+0000","last_clean":"2024-03-16T10:11:40.053703+0000","last_became_active":"2024-03-16T10:11:39.035382+0000","last_became_peered":"2024-03-16T10:11:39.035382+0000","last_unstale":"2024-03-16T10:11:40.053703+0000","last_undegraded":"2024-03-16T10:11:40.053703+0000","last_fullsized":"2024-03-16T10:11:40.053703+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:47.950316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.064246+0000","last_change":"2024-03-16T10:11:40.064246+0000","last_active":"2024-03-16T10:11:40.064246+0000","last_peered":"2024-03-16T10:11:40.064246+0000","last_clean":"2024-03-16T10:11:40.064246+0000","last_became_active":"2024-03-16T10:11:40.063529+0000","last_became_peered":"2024-03-16T10:11:40.063529+0000","last_unstale":"2024-03-16T10:11:40.064246+0000","last_undegraded":"2024-03-16T10:11:40.064246+0000","last_fullsized":"2024-03-16T10:11:40.064246+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:54.938794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"29'1","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045024+0000","last_change":"2024-03-16T10:11:39.046131+0000","last_active":"2024-03-16T10:11:40.045024+0000","last_peered":"2024-03-16T10:11:40.045024+0000","last_clean":"2024-03-16T10:11:40.045024+0000","last_became_active":"2024-03-16T10:11:39.045785+0000","last_became_peered":"2024-03-16T10:11:39.045785+0000","last_unstale":"2024-03-16T10:11:40.045024+0000","last_undegraded":"2024-03-16T10:11:40.045024+0000","last_fullsized":"2024-03-16T10:11:40.045024+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:48.922456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.037470+0000","last_change":"2024-03-16T10:11:40.037470+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.037470+0000","last_undegraded":"2024-03-16T10:11:40.037470+0000","last_fullsized":"2024-03-16T10:11:40.037470+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"29'2","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030277+0000","last_change":"2024-03-16T10:11:39.051711+0000","last_active":"2024-03-16T10:11:40.030277+0000","last_peered":"2024-03-16T10:11:40.030277+0000","last_clean":"2024-03-16T10:11:40.030277+0000","last_became_active":"2024-03-16T10:11:39.050887+0000","last_became_peered":"2024-03-16T10:11:39.050887+0000","last_unstale":"2024-03-16T10:11:40.030277+0000","last_undegraded":"2024-03-16T10:11:40.030277+0000","last_fullsized":"2024-03-16T10:11:40.030277+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:03:20.074611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.031547+0000","last_change":"2024-03-16T10:11:40.031547+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.031547+0000","last_undegraded":"2024-03-16T10:11:40.031547+0000","last_fullsized":"2024-03-16T10:11:40.031547+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":["1","4"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"29'1","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.026650+0000","last_change":"2024-03-16T10:11:39.059183+0000","last_active":"2024-03-16T10:11:40.026650+0000","last_peered":"2024-03-16T10:11:40.026650+0000","last_clean":"2024-03-16T10:11:40.026650+0000","last_became_active":"2024-03-16T10:11:39.052311+0000","last_became_peered":"2024-03-16T10:11:39.052311+0000","last_unstale":"2024-03-16T10:11:40.026650+0000","last_undegraded":"2024-03-16T10:11:40.026650+0000","last_fullsized":"2024-03-16T10:11:40.026650+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:28:22.985440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.024225+0000","last_change":"2024-03-16T10:11:40.024225+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.024225+0000","last_undegraded":"2024-03-16T10:11:40.024225+0000","last_fullsized":"2024-03-16T10:11:40.024225+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030295+0000","last_change":"2024-03-16T10:11:39.044835+0000","last_active":"2024-03-16T10:11:40.030295+0000","last_peered":"2024-03-16T10:11:40.030295+0000","last_clean":"2024-03-16T10:11:40.030295+0000","last_became_active":"2024-03-16T10:11:39.043928+0000","last_became_peered":"2024-03-16T10:11:39.043928+0000","last_unstale":"2024-03-16T10:11:40.030295+0000","last_undegraded":"2024-03-16T10:11:40.030295+0000","last_fullsized":"2024-03-16T10:11:40.030295+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:53.886085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.035827+0000","last_change":"2024-03-16T10:11:40.034188+0000","last_active":"2024-03-16T10:11:40.035827+0000","last_peered":"2024-03-16T10:11:40.035827+0000","last_clean":"2024-03-16T10:11:40.035827+0000","last_became_active":"2024-03-16T10:11:40.033873+0000","last_became_peered":"2024-03-16T10:11:40.033873+0000","last_unstale":"2024-03-16T10:11:40.035827+0000","last_undegraded":"2024-03-16T10:11:40.035827+0000","last_fullsized":"2024-03-16T10:11:40.035827+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:32:13.692871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"29'2","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.041749+0000","last_change":"2024-03-16T10:11:39.042466+0000","last_active":"2024-03-16T10:11:40.041749+0000","last_peered":"2024-03-16T10:11:40.041749+0000","last_clean":"2024-03-16T10:11:40.041749+0000","last_became_active":"2024-03-16T10:11:39.042100+0000","last_became_peered":"2024-03-16T10:11:39.042100+0000","last_unstale":"2024-03-16T10:11:40.041749+0000","last_undegraded":"2024-03-16T10:11:40.041749+0000","last_fullsized":"2024-03-16T10:11:40.041749+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:39.788886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032944+0000","last_change":"2024-03-16T10:11:40.031195+0000","last_active":"2024-03-16T10:11:40.032944+0000","last_peered":"2024-03-16T10:11:40.032944+0000","last_clean":"2024-03-16T10:11:40.032944+0000","last_became_active":"2024-03-16T10:11:40.030881+0000","last_became_peered":"2024-03-16T10:11:40.030881+0000","last_unstale":"2024-03-16T10:11:40.032944+0000","last_undegraded":"2024-03-16T10:11:40.032944+0000","last_fullsized":"2024-03-16T10:11:40.032944+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:46:51.399892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032725+0000","last_change":"2024-03-16T10:11:39.044909+0000","last_active":"2024-03-16T10:11:40.032725+0000","last_peered":"2024-03-16T10:11:40.032725+0000","last_clean":"2024-03-16T10:11:40.032725+0000","last_became_active":"2024-03-16T10:11:39.044151+0000","last_became_peered":"2024-03-16T10:11:39.044151+0000","last_unstale":"2024-03-16T10:11:40.032725+0000","last_undegraded":"2024-03-16T10:11:40.032725+0000","last_fullsized":"2024-03-16T10:11:40.032725+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032985+0000","last_change":"2024-03-16T10:11:40.031271+0000","last_active":"2024-03-16T10:11:40.032985+0000","last_peered":"2024-03-16T10:11:40.032985+0000","last_clean":"2024-03-16T10:11:40.032985+0000","last_became_active":"2024-03-16T10:11:40.030860+0000","last_became_peered":"2024-03-16T10:11:40.030860+0000","last_unstale":"2024-03-16T10:11:40.032985+0000","last_undegraded":"2024-03-16T10:11:40.032985+0000","last_fullsized":"2024-03-16T10:11:40.032985+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:32:50.216565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.030652+0000","last_change":"2024-03-16T10:11:39.059806+0000","last_active":"2024-03-16T10:11:40.030652+0000","last_peered":"2024-03-16T10:11:40.030652+0000","last_clean":"2024-03-16T10:11:40.030652+0000","last_became_active":"2024-03-16T10:11:39.053858+0000","last_became_peered":"2024-03-16T10:11:39.053858+0000","last_unstale":"2024-03-16T10:11:40.030652+0000","last_undegraded":"2024-03-16T10:11:40.030652+0000","last_fullsized":"2024-03-16T10:11:40.030652+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:15:51.282542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.039125+0000","last_change":"2024-03-16T10:11:40.039125+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.039125+0000","last_undegraded":"2024-03-16T10:11:40.039125+0000","last_fullsized":"2024-03-16T10:11:40.039125+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.036628+0000","last_change":"2024-03-16T10:11:39.051315+0000","last_active":"2024-03-16T10:11:40.036628+0000","last_peered":"2024-03-16T10:11:40.036628+0000","last_clean":"2024-03-16T10:11:40.036628+0000","last_became_active":"2024-03-16T10:11:39.050550+0000","last_became_peered":"2024-03-16T10:11:39.050550+0000","last_unstale":"2024-03-16T10:11:40.036628+0000","last_undegraded":"2024-03-16T10:11:40.036628+0000","last_fullsized":"2024-03-16T10:11:40.036628+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:11:55.739859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.024301+0000","last_change":"2024-03-16T10:11:40.024301+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.024301+0000","last_undegraded":"2024-03-16T10:11:40.024301+0000","last_fullsized":"2024-03-16T10:11:40.024301+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"29'2","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032987+0000","last_change":"2024-03-16T10:11:39.060338+0000","last_active":"2024-03-16T10:11:40.032987+0000","last_peered":"2024-03-16T10:11:40.032987+0000","last_clean":"2024-03-16T10:11:40.032987+0000","last_became_active":"2024-03-16T10:11:39.060040+0000","last_became_peered":"2024-03-16T10:11:39.060040+0000","last_unstale":"2024-03-16T10:11:40.032987+0000","last_undegraded":"2024-03-16T10:11:40.032987+0000","last_fullsized":"2024-03-16T10:11:40.032987+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:39.788886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.036465+0000","last_change":"2024-03-16T10:11:40.036465+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.036465+0000","last_undegraded":"2024-03-16T10:11:40.036465+0000","last_fullsized":"2024-03-16T10:11:40.036465+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027701+0000","last_change":"2024-03-16T10:11:39.042670+0000","last_active":"2024-03-16T10:11:40.027701+0000","last_peered":"2024-03-16T10:11:40.027701+0000","last_clean":"2024-03-16T10:11:40.027701+0000","last_became_active":"2024-03-16T10:11:39.042250+0000","last_became_peered":"2024-03-16T10:11:39.042250+0000","last_unstale":"2024-03-16T10:11:40.027701+0000","last_undegraded":"2024-03-16T10:11:40.027701+0000","last_fullsized":"2024-03-16T10:11:40.027701+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.036767+0000","last_change":"2024-03-16T10:11:40.035336+0000","last_active":"2024-03-16T10:11:40.036767+0000","last_peered":"2024-03-16T10:11:40.036767+0000","last_clean":"2024-03-16T10:11:40.036767+0000","last_became_active":"2024-03-16T10:11:40.035053+0000","last_became_peered":"2024-03-16T10:11:40.035053+0000","last_unstale":"2024-03-16T10:11:40.036767+0000","last_undegraded":"2024-03-16T10:11:40.036767+0000","last_fullsized":"2024-03-16T10:11:40.036767+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:30:54.938794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032840+0000","last_change":"2024-03-16T10:11:39.034786+0000","last_active":"2024-03-16T10:11:40.032840+0000","last_peered":"2024-03-16T10:11:40.032840+0000","last_clean":"2024-03-16T10:11:40.032840+0000","last_became_active":"2024-03-16T10:11:39.034431+0000","last_became_peered":"2024-03-16T10:11:39.034431+0000","last_unstale":"2024-03-16T10:11:40.032840+0000","last_undegraded":"2024-03-16T10:11:40.032840+0000","last_fullsized":"2024-03-16T10:11:40.032840+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.037552+0000","last_change":"2024-03-16T10:11:40.031811+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.037552+0000","last_undegraded":"2024-03-16T10:11:40.037552+0000","last_fullsized":"2024-03-16T10:11:40.037552+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.028056+0000","last_change":"2024-03-16T10:11:39.054972+0000","last_active":"2024-03-16T10:11:40.028056+0000","last_peered":"2024-03-16T10:11:40.028056+0000","last_clean":"2024-03-16T10:11:40.028056+0000","last_became_active":"2024-03-16T10:11:39.054554+0000","last_became_peered":"2024-03-16T10:11:39.054554+0000","last_unstale":"2024-03-16T10:11:40.028056+0000","last_undegraded":"2024-03-16T10:11:40.028056+0000","last_fullsized":"2024-03-16T10:11:40.028056+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:48.922456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":3,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.027157+0000","last_change":"2024-03-16T10:11:40.027157+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.027157+0000","last_undegraded":"2024-03-16T10:11:40.027157+0000","last_fullsized":"2024-03-16T10:11:40.027157+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.032789+0000","last_change":"2024-03-16T10:11:39.055099+0000","last_active":"2024-03-16T10:11:40.032789+0000","last_peered":"2024-03-16T10:11:40.032789+0000","last_clean":"2024-03-16T10:11:40.032789+0000","last_became_active":"2024-03-16T10:11:39.054743+0000","last_became_peered":"2024-03-16T10:11:39.054743+0000","last_unstale":"2024-03-16T10:11:40.032789+0000","last_undegraded":"2024-03-16T10:11:40.032789+0000","last_fullsized":"2024-03-16T10:11:40.032789+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049077+0000","last_change":"2024-03-16T10:11:40.045840+0000","last_active":"2024-03-16T10:11:40.049077+0000","last_peered":"2024-03-16T10:11:40.049077+0000","last_clean":"2024-03-16T10:11:40.049077+0000","last_became_active":"2024-03-16T10:11:40.040556+0000","last_became_peered":"2024-03-16T10:11:40.040556+0000","last_unstale":"2024-03-16T10:11:40.049077+0000","last_undegraded":"2024-03-16T10:11:40.049077+0000","last_fullsized":"2024-03-16T10:11:40.049077+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:44:53.636655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.031865+0000","last_change":"2024-03-16T10:11:39.053883+0000","last_active":"2024-03-16T10:11:40.031865+0000","last_peered":"2024-03-16T10:11:40.031865+0000","last_clean":"2024-03-16T10:11:40.031865+0000","last_became_active":"2024-03-16T10:11:39.052424+0000","last_became_peered":"2024-03-16T10:11:39.052424+0000","last_unstale":"2024-03-16T10:11:40.031865+0000","last_undegraded":"2024-03-16T10:11:40.031865+0000","last_fullsized":"2024-03-16T10:11:40.031865+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:35:59.698889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.049061+0000","last_change":"2024-03-16T10:11:40.049061+0000","last_active":"2024-03-16T10:11:40.049061+0000","last_peered":"2024-03-16T10:11:40.049061+0000","last_clean":"2024-03-16T10:11:40.049061+0000","last_became_active":"2024-03-16T10:11:40.048340+0000","last_became_peered":"2024-03-16T10:11:40.048340+0000","last_unstale":"2024-03-16T10:11:40.049061+0000","last_undegraded":"2024-03-16T10:11:40.049061+0000","last_fullsized":"2024-03-16T10:11:40.049061+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:21.380536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.034511+0000","last_change":"2024-03-16T10:11:39.049474+0000","last_active":"2024-03-16T10:11:40.034511+0000","last_peered":"2024-03-16T10:11:40.034511+0000","last_clean":"2024-03-16T10:11:40.034511+0000","last_became_active":"2024-03-16T10:11:39.049180+0000","last_became_peered":"2024-03-16T10:11:39.049180+0000","last_unstale":"2024-03-16T10:11:40.034511+0000","last_undegraded":"2024-03-16T10:11:40.034511+0000","last_fullsized":"2024-03-16T10:11:40.034511+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:34:23.568982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":4,"reported_epoch":29,"state":"creating+activating","last_fresh":"2024-03-16T10:11:40.045609+0000","last_change":"2024-03-16T10:11:40.039231+0000","last_active":"2024-03-16T10:11:39.007282+0000","last_peered":"2024-03-16T10:11:39.007282+0000","last_clean":"2024-03-16T10:11:39.007282+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T10:11:40.045609+0000","last_undegraded":"2024-03-16T10:11:40.045609+0000","last_fullsized":"2024-03-16T10:11:40.045609+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.027544+0000","last_change":"2024-03-16T10:11:39.055688+0000","last_active":"2024-03-16T10:11:40.027544+0000","last_peered":"2024-03-16T10:11:40.027544+0000","last_clean":"2024-03-16T10:11:40.027544+0000","last_became_active":"2024-03-16T10:11:39.054833+0000","last_became_peered":"2024-03-16T10:11:39.054833+0000","last_unstale":"2024-03-16T10:11:40.027544+0000","last_undegraded":"2024-03-16T10:11:40.027544+0000","last_fullsized":"2024-03-16T10:11:40.027544+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":185,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-16T10:11:40.045097+0000","last_change":"2024-03-16T10:10:48.248686+0000","last_active":"2024-03-16T10:11:40.045097+0000","last_peered":"2024-03-16T10:11:40.045097+0000","last_clean":"2024-03-16T10:11:40.045097+0000","last_became_active":"2024-03-16T10:10:48.243054+0000","last_became_peered":"2024-03-16T10:10:48.243054+0000","last_unstale":"2024-03-16T10:11:40.045097+0000","last_undegraded":"2024-03-16T10:11:40.045097+0000","last_fullsized":"2024-03-16T10:11:40.045097+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_clean_scrub_stamp":"2024-03-16T10:10:46.517938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:42.906473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1436,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":705976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27328,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344512,"statfs":{"total":96636764160,"available":96608780288,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":311296,"data_stored":115608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32064,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339776,"statfs":{"total":96636764160,"available":96603930624,"internally_reserved":0,"allocated":905216,"data_stored":705976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:11:47.737 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:11:47.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c903fb640 1 Processor -- start 2024-03-16T10:11:47.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c903fb640 1 -- start start 2024-03-16T10:11:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c903fb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b7650 0x7f8c881412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c903fb640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8c88141820 0x7f8c88151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c903fb640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c88141c50 con 0x7f8c880b4650 2024-03-16T10:11:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c903fb640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f8c880b4a30 con 0x7f8c88141820 2024-03-16T10:11:47.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c903fb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8c8814abe0 con 0x7f8c880b7650 2024-03-16T10:11:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c8d996640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b7650 0x7f8c881412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c8e998640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8c88141820 0x7f8c88151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c8d996640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b7650 0x7f8c881412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.17:56310/0 (socket says 172.21.15.17:56310) 2024-03-16T10:11:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c8d996640 1 -- 172.21.15.17:0/3459724492 learned_addr learned my addr 172.21.15.17:0/3459724492 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c8d996640 1 -- 172.21.15.17:0/3459724492 >> v2:172.21.15.174:3300/0 conn(0x7f8c88141820 msgr2=0x7f8c88151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c8d996640 1 --2- 172.21.15.17:0/3459724492 >> v2:172.21.15.174:3300/0 conn(0x7f8c88141820 0x7f8c88151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c8d996640 1 -- 172.21.15.17:0/3459724492 >> v1:172.21.15.174:6789/0 conn(0x7f8c880b4650 legacy=0x7f8c8814b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:47.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.864+0000 7f8c8d996640 1 -- 172.21.15.17:0/3459724492 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8814aec0 con 0x7f8c880b7650 2024-03-16T10:11:47.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c8d996640 1 --2- 172.21.15.17:0/3459724492 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b7650 0x7f8c881412e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009870 tx=0x7f8c7c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84d091a51ed3dc79 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:47.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c8d195640 1 -- 172.21.15.17:0/3459724492 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0b0070 con 0x7f8c880b7650 2024-03-16T10:11:47.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c8d195640 1 -- 172.21.15.17:0/3459724492 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c7c0ab600 con 0x7f8c880b7650 2024-03-16T10:11:47.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c8d195640 1 -- 172.21.15.17:0/3459724492 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0ab9d0 con 0x7f8c880b7650 2024-03-16T10:11:47.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 -- 172.21.15.17:0/3459724492 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b7650 msgr2=0x7f8c881412e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:47.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 --2- 172.21.15.17:0/3459724492 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b7650 0x7f8c881412e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009870 tx=0x7f8c7c01c4d0 comp rx=0 tx=0).stop 2024-03-16T10:11:47.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 -- 172.21.15.17:0/3459724492 shutdown_connections 2024-03-16T10:11:47.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 --2- 172.21.15.17:0/3459724492 >> v2:172.21.15.174:3300/0 conn(0x7f8c88141820 0x7f8c88151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:47.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 --2- 172.21.15.17:0/3459724492 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b7650 0x7f8c881412e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:47.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 -- 172.21.15.17:0/3459724492 >> 172.21.15.17:0/3459724492 conn(0x7f8c880bccd0 msgr2=0x7f8c880bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:47.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 -- 172.21.15.17:0/3459724492 shutdown_connections 2024-03-16T10:11:47.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 -- 172.21.15.17:0/3459724492 wait complete. 2024-03-16T10:11:47.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 Processor -- start 2024-03-16T10:11:47.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 -- start start 2024-03-16T10:11:47.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c880b4650 0x7f8c881dfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:47.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b7650 0x7f8c8813aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c88141820 0x7f8c88142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8c880b5b70 con 0x7f8c880b4650 2024-03-16T10:11:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c880b59f0 con 0x7f8c88141820 2024-03-16T10:11:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c903fb640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8c88154b30 con 0x7f8c880b7650 2024-03-16T10:11:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c8e197640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c880b4650 0x7f8c881dfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c8e197640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c880b4650 0x7f8c881dfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52554/0 (socket says 172.21.15.17:52554) 2024-03-16T10:11:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c8e197640 1 -- 172.21.15.17:0/2039295445 learned_addr learned my addr 172.21.15.17:0/2039295445 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c8d996640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b7650 0x7f8c8813aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.868+0000 7f8c8e998640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c88141820 0x7f8c88142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c8e197640 1 -- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b7650 msgr2=0x7f8c8813aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c8e197640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b7650 0x7f8c8813aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c8e197640 1 -- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c88141820 msgr2=0x7f8c88142860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c8e197640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c88141820 0x7f8c88142860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:47.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c8e197640 1 -- 172.21.15.17:0/2039295445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c881e02c0 con 0x7f8c880b4650 2024-03-16T10:11:47.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c8d996640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b7650 0x7f8c8813aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:47.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c8e197640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c880b4650 0x7f8c881dfca0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400c3a0 tx=0x7f8c8400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:47.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c7affd640 1 -- 172.21.15.17:0/2039295445 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c840052d0 con 0x7f8c880b4650 2024-03-16T10:11:47.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c7affd640 1 -- 172.21.15.17:0/2039295445 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c84005470 con 0x7f8c880b4650 2024-03-16T10:11:47.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c903fb640 1 -- 172.21.15.17:0/2039295445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8814bed0 con 0x7f8c880b4650 2024-03-16T10:11:47.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c7affd640 1 -- 172.21.15.17:0/2039295445 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84007260 con 0x7f8c880b4650 2024-03-16T10:11:47.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c903fb640 1 -- 172.21.15.17:0/2039295445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c88147f90 con 0x7f8c880b4650 2024-03-16T10:11:47.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c903fb640 1 -- 172.21.15.17:0/2039295445 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c88068360 con 0x7f8c880b4650 2024-03-16T10:11:47.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c7affd640 1 -- 172.21.15.17:0/2039295445 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8c84007400 con 0x7f8c880b4650 2024-03-16T10:11:47.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c7affd640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8c6c067660 0x7f8c6c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:47.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c7affd640 1 -- 172.21.15.17:0/2039295445 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f8c84084d20 con 0x7f8c880b4650 2024-03-16T10:11:47.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c8d996640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8c6c067660 0x7f8c6c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:47.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.872+0000 7f8c8d996640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8c6c067660 0x7f8c6c069b20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0ad000 tx=0x7f8c7c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:47.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:47.876+0000 7f8c7affd640 1 -- 172.21.15.17:0/2039295445 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8c840559b0 con 0x7f8c880b4650 2024-03-16T10:11:48.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.036+0000 7f8c903fb640 1 -- 172.21.15.17:0/2039295445 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c880b90c0 con 0x7f8c6c067660 2024-03-16T10:11:48.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.056+0000 7f8c7affd640 1 -- 172.21.15.17:0/2039295445 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251923 (secure 0 0 0) 0x7f8c880b90c0 con 0x7f8c6c067660 2024-03-16T10:11:48.063 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:48.064 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-16T10:11:48.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.056+0000 7f8c903fb640 1 -- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8c6c067660 msgr2=0x7f8c6c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:48.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.056+0000 7f8c903fb640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8c6c067660 0x7f8c6c069b20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0ad000 tx=0x7f8c7c008040 comp rx=0 tx=0).stop 2024-03-16T10:11:48.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.056+0000 7f8c903fb640 1 -- 172.21.15.17:0/2039295445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c880b4650 msgr2=0x7f8c881dfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:48.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.056+0000 7f8c903fb640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c880b4650 0x7f8c881dfca0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400c3a0 tx=0x7f8c8400c860 comp rx=0 tx=0).stop 2024-03-16T10:11:48.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.056+0000 7f8c903fb640 1 -- 172.21.15.17:0/2039295445 shutdown_connections 2024-03-16T10:11:48.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.056+0000 7f8c903fb640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8c6c067660 0x7f8c6c069b20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.056+0000 7f8c903fb640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c88141820 0x7f8c88142860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.056+0000 7f8c903fb640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c880b7650 0x7f8c8813aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.056+0000 7f8c903fb640 1 --2- 172.21.15.17:0/2039295445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c880b4650 0x7f8c881dfca0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.056+0000 7f8c903fb640 1 -- 172.21.15.17:0/2039295445 >> 172.21.15.17:0/2039295445 conn(0x7f8c880bccd0 msgr2=0x7f8c88140730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:48.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.056+0000 7f8c903fb640 1 -- 172.21.15.17:0/2039295445 shutdown_connections 2024-03-16T10:11:48.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.056+0000 7f8c903fb640 1 -- 172.21.15.17:0/2039295445 wait complete. 2024-03-16T10:11:48.080 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-03-16T10:11:48.031712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236656,"kb_used_data":3848,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738064,"statfs":{"total":773094113280,"available":772851777536,"internally_reserved":0,"allocated":3940352,"data_stored":2215020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.009488"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.028910+0000","last_change":"2024-03-16T10:11:40.063541+0000","last_active":"2024-03-16T10:11:41.028910+0000","last_peered":"2024-03-16T10:11:41.028910+0000","last_clean":"2024-03-16T10:11:41.028910+0000","last_became_active":"2024-03-16T10:11:40.062875+0000","last_became_peered":"2024-03-16T10:11:40.062875+0000","last_unstale":"2024-03-16T10:11:41.028910+0000","last_undegraded":"2024-03-16T10:11:41.028910+0000","last_fullsized":"2024-03-16T10:11:41.028910+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:48.954207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935830+0000","last_change":"2024-03-16T10:11:40.042825+0000","last_active":"2024-03-16T10:11:42.935830+0000","last_peered":"2024-03-16T10:11:42.935830+0000","last_clean":"2024-03-16T10:11:42.935830+0000","last_became_active":"2024-03-16T10:11:40.042533+0000","last_became_peered":"2024-03-16T10:11:40.042533+0000","last_unstale":"2024-03-16T10:11:42.935830+0000","last_undegraded":"2024-03-16T10:11:42.935830+0000","last_fullsized":"2024-03-16T10:11:42.935830+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:42:33.704421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.938193+0000","last_change":"2024-03-16T10:11:40.079042+0000","last_active":"2024-03-16T10:11:42.938193+0000","last_peered":"2024-03-16T10:11:42.938193+0000","last_clean":"2024-03-16T10:11:42.938193+0000","last_became_active":"2024-03-16T10:11:40.078426+0000","last_became_peered":"2024-03-16T10:11:40.078426+0000","last_unstale":"2024-03-16T10:11:42.938193+0000","last_undegraded":"2024-03-16T10:11:42.938193+0000","last_fullsized":"2024-03-16T10:11:42.938193+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:42:33.704421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.028081+0000","last_change":"2024-03-16T10:11:40.045382+0000","last_active":"2024-03-16T10:11:42.028081+0000","last_peered":"2024-03-16T10:11:42.028081+0000","last_clean":"2024-03-16T10:11:42.028081+0000","last_became_active":"2024-03-16T10:11:40.045183+0000","last_became_peered":"2024-03-16T10:11:40.045183+0000","last_unstale":"2024-03-16T10:11:42.028081+0000","last_undegraded":"2024-03-16T10:11:42.028081+0000","last_fullsized":"2024-03-16T10:11:42.028081+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:39:53.674994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.934751+0000","last_change":"2024-03-16T10:11:40.044298+0000","last_active":"2024-03-16T10:11:42.934751+0000","last_peered":"2024-03-16T10:11:42.934751+0000","last_clean":"2024-03-16T10:11:42.934751+0000","last_became_active":"2024-03-16T10:11:40.043984+0000","last_became_peered":"2024-03-16T10:11:40.043984+0000","last_unstale":"2024-03-16T10:11:42.934751+0000","last_undegraded":"2024-03-16T10:11:42.934751+0000","last_fullsized":"2024-03-16T10:11:42.934751+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:36:00.702779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935681+0000","last_change":"2024-03-16T10:11:40.074068+0000","last_active":"2024-03-16T10:11:42.935681+0000","last_peered":"2024-03-16T10:11:42.935681+0000","last_clean":"2024-03-16T10:11:42.935681+0000","last_became_active":"2024-03-16T10:11:40.073784+0000","last_became_peered":"2024-03-16T10:11:40.073784+0000","last_unstale":"2024-03-16T10:11:42.935681+0000","last_undegraded":"2024-03-16T10:11:42.935681+0000","last_fullsized":"2024-03-16T10:11:42.935681+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:53:20.689903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.264651+0000","last_change":"2024-03-16T10:11:40.074083+0000","last_active":"2024-03-16T10:11:41.264651+0000","last_peered":"2024-03-16T10:11:41.264651+0000","last_clean":"2024-03-16T10:11:41.264651+0000","last_became_active":"2024-03-16T10:11:40.073429+0000","last_became_peered":"2024-03-16T10:11:40.073429+0000","last_unstale":"2024-03-16T10:11:41.264651+0000","last_undegraded":"2024-03-16T10:11:41.264651+0000","last_fullsized":"2024-03-16T10:11:41.264651+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:15:34.084536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.263996+0000","last_change":"2024-03-16T10:11:40.056793+0000","last_active":"2024-03-16T10:11:41.263996+0000","last_peered":"2024-03-16T10:11:41.263996+0000","last_clean":"2024-03-16T10:11:41.263996+0000","last_became_active":"2024-03-16T10:11:40.055461+0000","last_became_peered":"2024-03-16T10:11:40.055461+0000","last_unstale":"2024-03-16T10:11:41.263996+0000","last_undegraded":"2024-03-16T10:11:41.263996+0000","last_fullsized":"2024-03-16T10:11:41.263996+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:06.536564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.029389+0000","last_change":"2024-03-16T10:11:40.047641+0000","last_active":"2024-03-16T10:11:41.029389+0000","last_peered":"2024-03-16T10:11:41.029389+0000","last_clean":"2024-03-16T10:11:41.029389+0000","last_became_active":"2024-03-16T10:11:40.046666+0000","last_became_peered":"2024-03-16T10:11:40.046666+0000","last_unstale":"2024-03-16T10:11:41.029389+0000","last_undegraded":"2024-03-16T10:11:41.029389+0000","last_fullsized":"2024-03-16T10:11:41.029389+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:39:53.674994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.028429+0000","last_change":"2024-03-16T10:11:40.047611+0000","last_active":"2024-03-16T10:11:42.028429+0000","last_peered":"2024-03-16T10:11:42.028429+0000","last_clean":"2024-03-16T10:11:42.028429+0000","last_became_active":"2024-03-16T10:11:40.047371+0000","last_became_peered":"2024-03-16T10:11:40.047371+0000","last_unstale":"2024-03-16T10:11:42.028429+0000","last_undegraded":"2024-03-16T10:11:42.028429+0000","last_fullsized":"2024-03-16T10:11:42.028429+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:35:29.488404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.264799+0000","last_change":"2024-03-16T10:11:40.060551+0000","last_active":"2024-03-16T10:11:41.264799+0000","last_peered":"2024-03-16T10:11:41.264799+0000","last_clean":"2024-03-16T10:11:41.264799+0000","last_became_active":"2024-03-16T10:11:40.059731+0000","last_became_peered":"2024-03-16T10:11:40.059731+0000","last_unstale":"2024-03-16T10:11:41.264799+0000","last_undegraded":"2024-03-16T10:11:41.264799+0000","last_fullsized":"2024-03-16T10:11:41.264799+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:42.385401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.937882+0000","last_change":"2024-03-16T10:11:40.085887+0000","last_active":"2024-03-16T10:11:42.937882+0000","last_peered":"2024-03-16T10:11:42.937882+0000","last_clean":"2024-03-16T10:11:42.937882+0000","last_became_active":"2024-03-16T10:11:40.085084+0000","last_became_peered":"2024-03-16T10:11:40.085084+0000","last_unstale":"2024-03-16T10:11:42.937882+0000","last_undegraded":"2024-03-16T10:11:42.937882+0000","last_fullsized":"2024-03-16T10:11:42.937882+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:53:20.689903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.934923+0000","last_change":"2024-03-16T10:11:40.038996+0000","last_active":"2024-03-16T10:11:42.934923+0000","last_peered":"2024-03-16T10:11:42.934923+0000","last_clean":"2024-03-16T10:11:42.934923+0000","last_became_active":"2024-03-16T10:11:40.036020+0000","last_became_peered":"2024-03-16T10:11:40.036020+0000","last_unstale":"2024-03-16T10:11:42.934923+0000","last_undegraded":"2024-03-16T10:11:42.934923+0000","last_fullsized":"2024-03-16T10:11:42.934923+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:58.221102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935881+0000","last_change":"2024-03-16T10:11:40.042699+0000","last_active":"2024-03-16T10:11:42.935881+0000","last_peered":"2024-03-16T10:11:42.935881+0000","last_clean":"2024-03-16T10:11:42.935881+0000","last_became_active":"2024-03-16T10:11:40.042299+0000","last_became_peered":"2024-03-16T10:11:40.042299+0000","last_unstale":"2024-03-16T10:11:42.935881+0000","last_undegraded":"2024-03-16T10:11:42.935881+0000","last_fullsized":"2024-03-16T10:11:42.935881+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:43:08.590085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.937554+0000","last_change":"2024-03-16T10:11:40.075980+0000","last_active":"2024-03-16T10:11:42.937554+0000","last_peered":"2024-03-16T10:11:42.937554+0000","last_clean":"2024-03-16T10:11:42.937554+0000","last_became_active":"2024-03-16T10:11:40.073901+0000","last_became_peered":"2024-03-16T10:11:40.073901+0000","last_unstale":"2024-03-16T10:11:42.937554+0000","last_undegraded":"2024-03-16T10:11:42.937554+0000","last_fullsized":"2024-03-16T10:11:42.937554+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:38:20.425271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.265178+0000","last_change":"2024-03-16T10:11:40.057307+0000","last_active":"2024-03-16T10:11:41.265178+0000","last_peered":"2024-03-16T10:11:41.265178+0000","last_clean":"2024-03-16T10:11:41.265178+0000","last_became_active":"2024-03-16T10:11:40.056150+0000","last_became_peered":"2024-03-16T10:11:40.056150+0000","last_unstale":"2024-03-16T10:11:41.265178+0000","last_undegraded":"2024-03-16T10:11:41.265178+0000","last_fullsized":"2024-03-16T10:11:41.265178+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:43:08.590085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.265077+0000","last_change":"2024-03-16T10:11:40.067933+0000","last_active":"2024-03-16T10:11:41.265077+0000","last_peered":"2024-03-16T10:11:41.265077+0000","last_clean":"2024-03-16T10:11:41.265077+0000","last_became_active":"2024-03-16T10:11:40.067168+0000","last_became_peered":"2024-03-16T10:11:40.067168+0000","last_unstale":"2024-03-16T10:11:41.265077+0000","last_undegraded":"2024-03-16T10:11:41.265077+0000","last_fullsized":"2024-03-16T10:11:41.265077+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:37:56.020398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.936394+0000","last_change":"2024-03-16T10:11:40.046723+0000","last_active":"2024-03-16T10:11:42.936394+0000","last_peered":"2024-03-16T10:11:42.936394+0000","last_clean":"2024-03-16T10:11:42.936394+0000","last_became_active":"2024-03-16T10:11:40.043586+0000","last_became_peered":"2024-03-16T10:11:40.043586+0000","last_unstale":"2024-03-16T10:11:42.936394+0000","last_undegraded":"2024-03-16T10:11:42.936394+0000","last_fullsized":"2024-03-16T10:11:42.936394+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:01.599220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935805+0000","last_change":"2024-03-16T10:11:40.043055+0000","last_active":"2024-03-16T10:11:42.935805+0000","last_peered":"2024-03-16T10:11:42.935805+0000","last_clean":"2024-03-16T10:11:42.935805+0000","last_became_active":"2024-03-16T10:11:40.041082+0000","last_became_peered":"2024-03-16T10:11:40.041082+0000","last_unstale":"2024-03-16T10:11:42.935805+0000","last_undegraded":"2024-03-16T10:11:42.935805+0000","last_fullsized":"2024-03-16T10:11:42.935805+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:15:34.084536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935074+0000","last_change":"2024-03-16T10:11:40.050291+0000","last_active":"2024-03-16T10:11:42.935074+0000","last_peered":"2024-03-16T10:11:42.935074+0000","last_clean":"2024-03-16T10:11:42.935074+0000","last_became_active":"2024-03-16T10:11:40.049981+0000","last_became_peered":"2024-03-16T10:11:40.049981+0000","last_unstale":"2024-03-16T10:11:42.935074+0000","last_undegraded":"2024-03-16T10:11:42.935074+0000","last_fullsized":"2024-03-16T10:11:42.935074+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:21.380536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.028282+0000","last_change":"2024-03-16T10:11:40.046160+0000","last_active":"2024-03-16T10:11:41.028282+0000","last_peered":"2024-03-16T10:11:41.028282+0000","last_clean":"2024-03-16T10:11:41.028282+0000","last_became_active":"2024-03-16T10:11:40.045457+0000","last_became_peered":"2024-03-16T10:11:40.045457+0000","last_unstale":"2024-03-16T10:11:41.028282+0000","last_undegraded":"2024-03-16T10:11:41.028282+0000","last_fullsized":"2024-03-16T10:11:41.028282+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:11:56.743750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.028465+0000","last_change":"2024-03-16T10:11:40.050020+0000","last_active":"2024-03-16T10:11:41.028465+0000","last_peered":"2024-03-16T10:11:41.028465+0000","last_clean":"2024-03-16T10:11:41.028465+0000","last_became_active":"2024-03-16T10:11:40.047232+0000","last_became_peered":"2024-03-16T10:11:40.047232+0000","last_unstale":"2024-03-16T10:11:41.028465+0000","last_undegraded":"2024-03-16T10:11:41.028465+0000","last_fullsized":"2024-03-16T10:11:41.028465+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:19.819589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.025520+0000","last_change":"2024-03-16T10:11:40.060551+0000","last_active":"2024-03-16T10:11:41.025520+0000","last_peered":"2024-03-16T10:11:41.025520+0000","last_clean":"2024-03-16T10:11:41.025520+0000","last_became_active":"2024-03-16T10:11:40.060260+0000","last_became_peered":"2024-03-16T10:11:40.060260+0000","last_unstale":"2024-03-16T10:11:41.025520+0000","last_undegraded":"2024-03-16T10:11:41.025520+0000","last_fullsized":"2024-03-16T10:11:41.025520+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:45:21.820367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.937460+0000","last_change":"2024-03-16T10:11:40.041024+0000","last_active":"2024-03-16T10:11:42.937460+0000","last_peered":"2024-03-16T10:11:42.937460+0000","last_clean":"2024-03-16T10:11:42.937460+0000","last_became_active":"2024-03-16T10:11:40.040230+0000","last_became_peered":"2024-03-16T10:11:40.040230+0000","last_unstale":"2024-03-16T10:11:42.937460+0000","last_undegraded":"2024-03-16T10:11:42.937460+0000","last_fullsized":"2024-03-16T10:11:42.937460+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:55:18.201863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.028699+0000","last_change":"2024-03-16T10:11:40.036983+0000","last_active":"2024-03-16T10:11:42.028699+0000","last_peered":"2024-03-16T10:11:42.028699+0000","last_clean":"2024-03-16T10:11:42.028699+0000","last_became_active":"2024-03-16T10:11:40.036701+0000","last_became_peered":"2024-03-16T10:11:40.036701+0000","last_unstale":"2024-03-16T10:11:42.028699+0000","last_undegraded":"2024-03-16T10:11:42.028699+0000","last_fullsized":"2024-03-16T10:11:42.028699+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:56:04.173993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.936394+0000","last_change":"2024-03-16T10:11:40.059380+0000","last_active":"2024-03-16T10:11:42.936394+0000","last_peered":"2024-03-16T10:11:42.936394+0000","last_clean":"2024-03-16T10:11:42.936394+0000","last_became_active":"2024-03-16T10:11:40.059069+0000","last_became_peered":"2024-03-16T10:11:40.059069+0000","last_unstale":"2024-03-16T10:11:42.936394+0000","last_undegraded":"2024-03-16T10:11:42.936394+0000","last_fullsized":"2024-03-16T10:11:42.936394+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:40:19.978624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.028380+0000","last_change":"2024-03-16T10:11:40.046735+0000","last_active":"2024-03-16T10:11:42.028380+0000","last_peered":"2024-03-16T10:11:42.028380+0000","last_clean":"2024-03-16T10:11:42.028380+0000","last_became_active":"2024-03-16T10:11:40.046202+0000","last_became_peered":"2024-03-16T10:11:40.046202+0000","last_unstale":"2024-03-16T10:11:42.028380+0000","last_undegraded":"2024-03-16T10:11:42.028380+0000","last_fullsized":"2024-03-16T10:11:42.028380+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:36:55.327282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.938010+0000","last_change":"2024-03-16T10:11:40.081390+0000","last_active":"2024-03-16T10:11:42.938010+0000","last_peered":"2024-03-16T10:11:42.938010+0000","last_clean":"2024-03-16T10:11:42.938010+0000","last_became_active":"2024-03-16T10:11:40.080646+0000","last_became_peered":"2024-03-16T10:11:40.080646+0000","last_unstale":"2024-03-16T10:11:42.938010+0000","last_undegraded":"2024-03-16T10:11:42.938010+0000","last_fullsized":"2024-03-16T10:11:42.938010+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:37:56.020398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.028728+0000","last_change":"2024-03-16T10:11:40.067258+0000","last_active":"2024-03-16T10:11:41.028728+0000","last_peered":"2024-03-16T10:11:41.028728+0000","last_clean":"2024-03-16T10:11:41.028728+0000","last_became_active":"2024-03-16T10:11:40.066423+0000","last_became_peered":"2024-03-16T10:11:40.066423+0000","last_unstale":"2024-03-16T10:11:41.028728+0000","last_undegraded":"2024-03-16T10:11:41.028728+0000","last_fullsized":"2024-03-16T10:11:41.028728+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:09.233109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.264371+0000","last_change":"2024-03-16T10:11:40.068815+0000","last_active":"2024-03-16T10:11:41.264371+0000","last_peered":"2024-03-16T10:11:41.264371+0000","last_clean":"2024-03-16T10:11:41.264371+0000","last_became_active":"2024-03-16T10:11:40.067533+0000","last_became_peered":"2024-03-16T10:11:40.067533+0000","last_unstale":"2024-03-16T10:11:41.264371+0000","last_undegraded":"2024-03-16T10:11:41.264371+0000","last_fullsized":"2024-03-16T10:11:41.264371+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:53:20.689903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.030549+0000","last_change":"2024-03-16T10:11:40.072126+0000","last_active":"2024-03-16T10:11:41.030549+0000","last_peered":"2024-03-16T10:11:41.030549+0000","last_clean":"2024-03-16T10:11:41.030549+0000","last_became_active":"2024-03-16T10:11:40.059182+0000","last_became_peered":"2024-03-16T10:11:40.059182+0000","last_unstale":"2024-03-16T10:11:41.030549+0000","last_undegraded":"2024-03-16T10:11:41.030549+0000","last_fullsized":"2024-03-16T10:11:41.030549+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:11:56.743750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.264236+0000","last_change":"2024-03-16T10:11:40.081733+0000","last_active":"2024-03-16T10:11:41.264236+0000","last_peered":"2024-03-16T10:11:41.264236+0000","last_clean":"2024-03-16T10:11:41.264236+0000","last_became_active":"2024-03-16T10:11:40.080557+0000","last_became_peered":"2024-03-16T10:11:40.080557+0000","last_unstale":"2024-03-16T10:11:41.264236+0000","last_undegraded":"2024-03-16T10:11:41.264236+0000","last_fullsized":"2024-03-16T10:11:41.264236+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:01:38.618684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.026103+0000","last_change":"2024-03-16T10:11:40.059343+0000","last_active":"2024-03-16T10:11:41.026103+0000","last_peered":"2024-03-16T10:11:41.026103+0000","last_clean":"2024-03-16T10:11:41.026103+0000","last_became_active":"2024-03-16T10:11:40.059049+0000","last_became_peered":"2024-03-16T10:11:40.059049+0000","last_unstale":"2024-03-16T10:11:41.026103+0000","last_undegraded":"2024-03-16T10:11:41.026103+0000","last_fullsized":"2024-03-16T10:11:41.026103+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:09.233109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"29'4","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.029549+0000","last_change":"2024-03-16T10:11:39.061998+0000","last_active":"2024-03-16T10:11:41.029549+0000","last_peered":"2024-03-16T10:11:41.029549+0000","last_clean":"2024-03-16T10:11:41.029549+0000","last_became_active":"2024-03-16T10:11:39.061249+0000","last_became_peered":"2024-03-16T10:11:39.061249+0000","last_unstale":"2024-03-16T10:11:41.029549+0000","last_undegraded":"2024-03-16T10:11:41.029549+0000","last_fullsized":"2024-03-16T10:11:41.029549+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.029311+0000","last_change":"2024-03-16T10:11:40.061676+0000","last_active":"2024-03-16T10:11:41.029311+0000","last_peered":"2024-03-16T10:11:41.029311+0000","last_clean":"2024-03-16T10:11:41.029311+0000","last_became_active":"2024-03-16T10:11:40.060865+0000","last_became_peered":"2024-03-16T10:11:40.060865+0000","last_unstale":"2024-03-16T10:11:41.029311+0000","last_undegraded":"2024-03-16T10:11:41.029311+0000","last_fullsized":"2024-03-16T10:11:41.029311+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:34:24.572872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.026157+0000","last_change":"2024-03-16T10:11:39.048117+0000","last_active":"2024-03-16T10:11:41.026157+0000","last_peered":"2024-03-16T10:11:41.026157+0000","last_clean":"2024-03-16T10:11:41.026157+0000","last_became_active":"2024-03-16T10:11:39.047798+0000","last_became_peered":"2024-03-16T10:11:39.047798+0000","last_unstale":"2024-03-16T10:11:41.026157+0000","last_undegraded":"2024-03-16T10:11:41.026157+0000","last_fullsized":"2024-03-16T10:11:41.026157+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.028783+0000","last_change":"2024-03-16T10:11:40.062870+0000","last_active":"2024-03-16T10:11:41.028783+0000","last_peered":"2024-03-16T10:11:41.028783+0000","last_clean":"2024-03-16T10:11:41.028783+0000","last_became_active":"2024-03-16T10:11:40.054629+0000","last_became_peered":"2024-03-16T10:11:40.054629+0000","last_unstale":"2024-03-16T10:11:41.028783+0000","last_undegraded":"2024-03-16T10:11:41.028783+0000","last_fullsized":"2024-03-16T10:11:41.028783+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:38:40.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935811+0000","last_change":"2024-03-16T10:11:39.038358+0000","last_active":"2024-03-16T10:11:42.935811+0000","last_peered":"2024-03-16T10:11:42.935811+0000","last_clean":"2024-03-16T10:11:42.935811+0000","last_became_active":"2024-03-16T10:11:39.038055+0000","last_became_peered":"2024-03-16T10:11:39.038055+0000","last_unstale":"2024-03-16T10:11:42.935811+0000","last_undegraded":"2024-03-16T10:11:42.935811+0000","last_fullsized":"2024-03-16T10:11:42.935811+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:11:55.739859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.028524+0000","last_change":"2024-03-16T10:11:40.046475+0000","last_active":"2024-03-16T10:11:42.028524+0000","last_peered":"2024-03-16T10:11:42.028524+0000","last_clean":"2024-03-16T10:11:42.028524+0000","last_became_active":"2024-03-16T10:11:40.045876+0000","last_became_peered":"2024-03-16T10:11:40.045876+0000","last_unstale":"2024-03-16T10:11:42.028524+0000","last_undegraded":"2024-03-16T10:11:42.028524+0000","last_fullsized":"2024-03-16T10:11:42.028524+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:58:06.536564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935128+0000","last_change":"2024-03-16T10:11:39.030352+0000","last_active":"2024-03-16T10:11:42.935128+0000","last_peered":"2024-03-16T10:11:42.935128+0000","last_clean":"2024-03-16T10:11:42.935128+0000","last_became_active":"2024-03-16T10:11:39.029031+0000","last_became_peered":"2024-03-16T10:11:39.029031+0000","last_unstale":"2024-03-16T10:11:42.935128+0000","last_undegraded":"2024-03-16T10:11:42.935128+0000","last_fullsized":"2024-03-16T10:11:42.935128+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:07:13.816782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.264350+0000","last_change":"2024-03-16T10:11:40.080755+0000","last_active":"2024-03-16T10:11:41.264350+0000","last_peered":"2024-03-16T10:11:41.264350+0000","last_clean":"2024-03-16T10:11:41.264350+0000","last_became_active":"2024-03-16T10:11:40.080175+0000","last_became_peered":"2024-03-16T10:11:40.080175+0000","last_unstale":"2024-03-16T10:11:41.264350+0000","last_undegraded":"2024-03-16T10:11:41.264350+0000","last_fullsized":"2024-03-16T10:11:41.264350+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:55:18.201863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.028969+0000","last_change":"2024-03-16T10:11:39.061267+0000","last_active":"2024-03-16T10:11:41.028969+0000","last_peered":"2024-03-16T10:11:41.028969+0000","last_clean":"2024-03-16T10:11:41.028969+0000","last_became_active":"2024-03-16T10:11:39.060323+0000","last_became_peered":"2024-03-16T10:11:39.060323+0000","last_unstale":"2024-03-16T10:11:41.028969+0000","last_undegraded":"2024-03-16T10:11:41.028969+0000","last_fullsized":"2024-03-16T10:11:41.028969+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:33:59.939254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.938012+0000","last_change":"2024-03-16T10:11:40.078354+0000","last_active":"2024-03-16T10:11:42.938012+0000","last_peered":"2024-03-16T10:11:42.938012+0000","last_clean":"2024-03-16T10:11:42.938012+0000","last_became_active":"2024-03-16T10:11:40.077704+0000","last_became_peered":"2024-03-16T10:11:40.077704+0000","last_unstale":"2024-03-16T10:11:42.938012+0000","last_undegraded":"2024-03-16T10:11:42.938012+0000","last_fullsized":"2024-03-16T10:11:42.938012+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:15:34.084536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.029389+0000","last_change":"2024-03-16T10:11:39.061259+0000","last_active":"2024-03-16T10:11:41.029389+0000","last_peered":"2024-03-16T10:11:41.029389+0000","last_clean":"2024-03-16T10:11:41.029389+0000","last_became_active":"2024-03-16T10:11:39.060484+0000","last_became_peered":"2024-03-16T10:11:39.060484+0000","last_unstale":"2024-03-16T10:11:41.029389+0000","last_undegraded":"2024-03-16T10:11:41.029389+0000","last_fullsized":"2024-03-16T10:11:41.029389+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:53:54.254670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.027740+0000","last_change":"2024-03-16T10:11:40.071418+0000","last_active":"2024-03-16T10:11:41.027740+0000","last_peered":"2024-03-16T10:11:41.027740+0000","last_clean":"2024-03-16T10:11:41.027740+0000","last_became_active":"2024-03-16T10:11:40.070642+0000","last_became_peered":"2024-03-16T10:11:40.070642+0000","last_unstale":"2024-03-16T10:11:41.027740+0000","last_undegraded":"2024-03-16T10:11:41.027740+0000","last_fullsized":"2024-03-16T10:11:41.027740+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:34:24.572872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935573+0000","last_change":"2024-03-16T10:11:39.033585+0000","last_active":"2024-03-16T10:11:42.935573+0000","last_peered":"2024-03-16T10:11:42.935573+0000","last_clean":"2024-03-16T10:11:42.935573+0000","last_became_active":"2024-03-16T10:11:39.033131+0000","last_became_peered":"2024-03-16T10:11:39.033131+0000","last_unstale":"2024-03-16T10:11:42.935573+0000","last_undegraded":"2024-03-16T10:11:42.935573+0000","last_fullsized":"2024-03-16T10:11:42.935573+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:35:42.208648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.028500+0000","last_change":"2024-03-16T10:11:40.060809+0000","last_active":"2024-03-16T10:11:42.028500+0000","last_peered":"2024-03-16T10:11:42.028500+0000","last_clean":"2024-03-16T10:11:42.028500+0000","last_became_active":"2024-03-16T10:11:40.060477+0000","last_became_peered":"2024-03-16T10:11:40.060477+0000","last_unstale":"2024-03-16T10:11:42.028500+0000","last_undegraded":"2024-03-16T10:11:42.028500+0000","last_fullsized":"2024-03-16T10:11:42.028500+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:44:53.636655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.030373+0000","last_change":"2024-03-16T10:11:39.053374+0000","last_active":"2024-03-16T10:11:41.030373+0000","last_peered":"2024-03-16T10:11:41.030373+0000","last_clean":"2024-03-16T10:11:41.030373+0000","last_became_active":"2024-03-16T10:11:39.051683+0000","last_became_peered":"2024-03-16T10:11:39.051683+0000","last_unstale":"2024-03-16T10:11:41.030373+0000","last_undegraded":"2024-03-16T10:11:41.030373+0000","last_fullsized":"2024-03-16T10:11:41.030373+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:08:49.668710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.938574+0000","last_change":"2024-03-16T10:11:40.086805+0000","last_active":"2024-03-16T10:11:42.938574+0000","last_peered":"2024-03-16T10:11:42.938574+0000","last_clean":"2024-03-16T10:11:42.938574+0000","last_became_active":"2024-03-16T10:11:40.086206+0000","last_became_peered":"2024-03-16T10:11:40.086206+0000","last_unstale":"2024-03-16T10:11:42.938574+0000","last_undegraded":"2024-03-16T10:11:42.938574+0000","last_fullsized":"2024-03-16T10:11:42.938574+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T11:40:19.978624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.936454+0000","last_change":"2024-03-16T10:11:39.050941+0000","last_active":"2024-03-16T10:11:42.936454+0000","last_peered":"2024-03-16T10:11:42.936454+0000","last_clean":"2024-03-16T10:11:42.936454+0000","last_became_active":"2024-03-16T10:11:39.050659+0000","last_became_peered":"2024-03-16T10:11:39.050659+0000","last_unstale":"2024-03-16T10:11:42.936454+0000","last_undegraded":"2024-03-16T10:11:42.936454+0000","last_fullsized":"2024-03-16T10:11:42.936454+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:08.229218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.936434+0000","last_change":"2024-03-16T10:11:40.043035+0000","last_active":"2024-03-16T10:11:42.936434+0000","last_peered":"2024-03-16T10:11:42.936434+0000","last_clean":"2024-03-16T10:11:42.936434+0000","last_became_active":"2024-03-16T10:11:40.040585+0000","last_became_peered":"2024-03-16T10:11:40.040585+0000","last_unstale":"2024-03-16T10:11:42.936434+0000","last_undegraded":"2024-03-16T10:11:42.936434+0000","last_fullsized":"2024-03-16T10:11:42.936434+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:05:50.963319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.026352+0000","last_change":"2024-03-16T10:11:39.045927+0000","last_active":"2024-03-16T10:11:41.026352+0000","last_peered":"2024-03-16T10:11:41.026352+0000","last_clean":"2024-03-16T10:11:41.026352+0000","last_became_active":"2024-03-16T10:11:39.045502+0000","last_became_peered":"2024-03-16T10:11:39.045502+0000","last_unstale":"2024-03-16T10:11:41.026352+0000","last_undegraded":"2024-03-16T10:11:41.026352+0000","last_fullsized":"2024-03-16T10:11:41.026352+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.936010+0000","last_change":"2024-03-16T10:11:40.041235+0000","last_active":"2024-03-16T10:11:42.936010+0000","last_peered":"2024-03-16T10:11:42.936010+0000","last_clean":"2024-03-16T10:11:42.936010+0000","last_became_active":"2024-03-16T10:11:40.040856+0000","last_became_peered":"2024-03-16T10:11:40.040856+0000","last_unstale":"2024-03-16T10:11:42.936010+0000","last_undegraded":"2024-03-16T10:11:42.936010+0000","last_fullsized":"2024-03-16T10:11:42.936010+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:37:56.020398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"29'3","reported_seq":17,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.938462+0000","last_change":"2024-03-16T10:11:39.052193+0000","last_active":"2024-03-16T10:11:42.938462+0000","last_peered":"2024-03-16T10:11:42.938462+0000","last_clean":"2024-03-16T10:11:42.938462+0000","last_became_active":"2024-03-16T10:11:39.050633+0000","last_became_peered":"2024-03-16T10:11:39.050633+0000","last_unstale":"2024-03-16T10:11:42.938462+0000","last_undegraded":"2024-03-16T10:11:42.938462+0000","last_fullsized":"2024-03-16T10:11:42.938462+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:55:08.229218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.938189+0000","last_change":"2024-03-16T10:11:40.078959+0000","last_active":"2024-03-16T10:11:42.938189+0000","last_peered":"2024-03-16T10:11:42.938189+0000","last_clean":"2024-03-16T10:11:42.938189+0000","last_became_active":"2024-03-16T10:11:40.077642+0000","last_became_peered":"2024-03-16T10:11:40.077642+0000","last_unstale":"2024-03-16T10:11:42.938189+0000","last_undegraded":"2024-03-16T10:11:42.938189+0000","last_fullsized":"2024-03-16T10:11:42.938189+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:42.385401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"29'6","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.936179+0000","last_change":"2024-03-16T10:11:39.042282+0000","last_active":"2024-03-16T10:11:42.936179+0000","last_peered":"2024-03-16T10:11:42.936179+0000","last_clean":"2024-03-16T10:11:42.936179+0000","last_became_active":"2024-03-16T10:11:39.041924+0000","last_became_peered":"2024-03-16T10:11:39.041924+0000","last_unstale":"2024-03-16T10:11:42.936179+0000","last_undegraded":"2024-03-16T10:11:42.936179+0000","last_fullsized":"2024-03-16T10:11:42.936179+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:18.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.027899+0000","last_change":"2024-03-16T10:11:40.073585+0000","last_active":"2024-03-16T10:11:42.027899+0000","last_peered":"2024-03-16T10:11:42.027899+0000","last_clean":"2024-03-16T10:11:42.027899+0000","last_became_active":"2024-03-16T10:11:40.073261+0000","last_became_peered":"2024-03-16T10:11:40.073261+0000","last_unstale":"2024-03-16T10:11:42.027899+0000","last_undegraded":"2024-03-16T10:11:42.027899+0000","last_fullsized":"2024-03-16T10:11:42.027899+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:38:40.506534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.029279+0000","last_change":"2024-03-16T10:11:39.042217+0000","last_active":"2024-03-16T10:11:41.029279+0000","last_peered":"2024-03-16T10:11:41.029279+0000","last_clean":"2024-03-16T10:11:41.029279+0000","last_became_active":"2024-03-16T10:11:39.041514+0000","last_became_peered":"2024-03-16T10:11:39.041514+0000","last_unstale":"2024-03-16T10:11:41.029279+0000","last_undegraded":"2024-03-16T10:11:41.029279+0000","last_fullsized":"2024-03-16T10:11:41.029279+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:12:12.889055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935890+0000","last_change":"2024-03-16T10:11:40.048743+0000","last_active":"2024-03-16T10:11:42.935890+0000","last_peered":"2024-03-16T10:11:42.935890+0000","last_clean":"2024-03-16T10:11:42.935890+0000","last_became_active":"2024-03-16T10:11:40.047008+0000","last_became_peered":"2024-03-16T10:11:40.047008+0000","last_unstale":"2024-03-16T10:11:42.935890+0000","last_undegraded":"2024-03-16T10:11:42.935890+0000","last_fullsized":"2024-03-16T10:11:42.935890+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:17:42.385401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.028033+0000","last_change":"2024-03-16T10:11:39.047491+0000","last_active":"2024-03-16T10:11:42.028033+0000","last_peered":"2024-03-16T10:11:42.028033+0000","last_clean":"2024-03-16T10:11:42.028033+0000","last_became_active":"2024-03-16T10:11:39.047085+0000","last_became_peered":"2024-03-16T10:11:39.047085+0000","last_unstale":"2024-03-16T10:11:42.028033+0000","last_undegraded":"2024-03-16T10:11:42.028033+0000","last_fullsized":"2024-03-16T10:11:42.028033+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.264703+0000","last_change":"2024-03-16T10:11:40.082127+0000","last_active":"2024-03-16T10:11:41.264703+0000","last_peered":"2024-03-16T10:11:41.264703+0000","last_clean":"2024-03-16T10:11:41.264703+0000","last_became_active":"2024-03-16T10:11:40.081371+0000","last_became_peered":"2024-03-16T10:11:40.081371+0000","last_unstale":"2024-03-16T10:11:41.264703+0000","last_undegraded":"2024-03-16T10:11:41.264703+0000","last_fullsized":"2024-03-16T10:11:41.264703+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:42:33.704421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935696+0000","last_change":"2024-03-16T10:11:39.053564+0000","last_active":"2024-03-16T10:11:42.935696+0000","last_peered":"2024-03-16T10:11:42.935696+0000","last_clean":"2024-03-16T10:11:42.935696+0000","last_became_active":"2024-03-16T10:11:39.053183+0000","last_became_peered":"2024-03-16T10:11:39.053183+0000","last_unstale":"2024-03-16T10:11:42.935696+0000","last_undegraded":"2024-03-16T10:11:42.935696+0000","last_fullsized":"2024-03-16T10:11:42.935696+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:45:20.816477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.030833+0000","last_change":"2024-03-16T10:11:40.059622+0000","last_active":"2024-03-16T10:11:41.030833+0000","last_peered":"2024-03-16T10:11:41.030833+0000","last_clean":"2024-03-16T10:11:41.030833+0000","last_became_active":"2024-03-16T10:11:40.058392+0000","last_became_peered":"2024-03-16T10:11:40.058392+0000","last_unstale":"2024-03-16T10:11:41.030833+0000","last_undegraded":"2024-03-16T10:11:41.030833+0000","last_fullsized":"2024-03-16T10:11:41.030833+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:19.819589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"29'8","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935540+0000","last_change":"2024-03-16T10:11:39.035694+0000","last_active":"2024-03-16T10:11:42.935540+0000","last_peered":"2024-03-16T10:11:42.935540+0000","last_clean":"2024-03-16T10:11:42.935540+0000","last_became_active":"2024-03-16T10:11:39.035382+0000","last_became_peered":"2024-03-16T10:11:39.035382+0000","last_unstale":"2024-03-16T10:11:42.935540+0000","last_undegraded":"2024-03-16T10:11:42.935540+0000","last_fullsized":"2024-03-16T10:11:42.935540+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:38:47.950316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.029103+0000","last_change":"2024-03-16T10:11:40.064246+0000","last_active":"2024-03-16T10:11:41.029103+0000","last_peered":"2024-03-16T10:11:41.029103+0000","last_clean":"2024-03-16T10:11:41.029103+0000","last_became_active":"2024-03-16T10:11:40.063529+0000","last_became_peered":"2024-03-16T10:11:40.063529+0000","last_unstale":"2024-03-16T10:11:41.029103+0000","last_undegraded":"2024-03-16T10:11:41.029103+0000","last_fullsized":"2024-03-16T10:11:41.029103+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:09:41.244065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.028926+0000","last_change":"2024-03-16T10:11:39.046131+0000","last_active":"2024-03-16T10:11:42.028926+0000","last_peered":"2024-03-16T10:11:42.028926+0000","last_clean":"2024-03-16T10:11:42.028926+0000","last_became_active":"2024-03-16T10:11:39.045785+0000","last_became_peered":"2024-03-16T10:11:39.045785+0000","last_unstale":"2024-03-16T10:11:42.028926+0000","last_undegraded":"2024-03-16T10:11:42.028926+0000","last_fullsized":"2024-03-16T10:11:42.028926+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:48.922456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.938422+0000","last_change":"2024-03-16T10:11:40.080973+0000","last_active":"2024-03-16T10:11:42.938422+0000","last_peered":"2024-03-16T10:11:42.938422+0000","last_clean":"2024-03-16T10:11:42.938422+0000","last_became_active":"2024-03-16T10:11:40.080362+0000","last_became_peered":"2024-03-16T10:11:40.080362+0000","last_unstale":"2024-03-16T10:11:42.938422+0000","last_undegraded":"2024-03-16T10:11:42.938422+0000","last_fullsized":"2024-03-16T10:11:42.938422+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:43:08.590085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.029844+0000","last_change":"2024-03-16T10:11:39.051711+0000","last_active":"2024-03-16T10:11:41.029844+0000","last_peered":"2024-03-16T10:11:41.029844+0000","last_clean":"2024-03-16T10:11:41.029844+0000","last_became_active":"2024-03-16T10:11:39.050887+0000","last_became_peered":"2024-03-16T10:11:39.050887+0000","last_unstale":"2024-03-16T10:11:41.029844+0000","last_undegraded":"2024-03-16T10:11:41.029844+0000","last_fullsized":"2024-03-16T10:11:41.029844+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:03:20.074611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.028167+0000","last_change":"2024-03-16T10:11:40.079642+0000","last_active":"2024-03-16T10:11:42.028167+0000","last_peered":"2024-03-16T10:11:42.028167+0000","last_clean":"2024-03-16T10:11:42.028167+0000","last_became_active":"2024-03-16T10:11:40.079397+0000","last_became_peered":"2024-03-16T10:11:40.079397+0000","last_unstale":"2024-03-16T10:11:42.028167+0000","last_undegraded":"2024-03-16T10:11:42.028167+0000","last_fullsized":"2024-03-16T10:11:42.028167+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:09:41.244065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.028612+0000","last_change":"2024-03-16T10:11:39.059183+0000","last_active":"2024-03-16T10:11:41.028612+0000","last_peered":"2024-03-16T10:11:41.028612+0000","last_clean":"2024-03-16T10:11:41.028612+0000","last_became_active":"2024-03-16T10:11:39.052311+0000","last_became_peered":"2024-03-16T10:11:39.052311+0000","last_unstale":"2024-03-16T10:11:41.028612+0000","last_undegraded":"2024-03-16T10:11:41.028612+0000","last_fullsized":"2024-03-16T10:11:41.028612+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:28:22.985440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935101+0000","last_change":"2024-03-16T10:11:40.052124+0000","last_active":"2024-03-16T10:11:42.935101+0000","last_peered":"2024-03-16T10:11:42.935101+0000","last_clean":"2024-03-16T10:11:42.935101+0000","last_became_active":"2024-03-16T10:11:40.051786+0000","last_became_peered":"2024-03-16T10:11:40.051786+0000","last_unstale":"2024-03-16T10:11:42.935101+0000","last_undegraded":"2024-03-16T10:11:42.935101+0000","last_fullsized":"2024-03-16T10:11:42.935101+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:49.926346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.029035+0000","last_change":"2024-03-16T10:11:39.044835+0000","last_active":"2024-03-16T10:11:41.029035+0000","last_peered":"2024-03-16T10:11:41.029035+0000","last_clean":"2024-03-16T10:11:41.029035+0000","last_became_active":"2024-03-16T10:11:39.043928+0000","last_became_peered":"2024-03-16T10:11:39.043928+0000","last_unstale":"2024-03-16T10:11:41.029035+0000","last_undegraded":"2024-03-16T10:11:41.029035+0000","last_fullsized":"2024-03-16T10:11:41.029035+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:31:53.886085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.934753+0000","last_change":"2024-03-16T10:11:40.034188+0000","last_active":"2024-03-16T10:11:42.934753+0000","last_peered":"2024-03-16T10:11:42.934753+0000","last_clean":"2024-03-16T10:11:42.934753+0000","last_became_active":"2024-03-16T10:11:40.033873+0000","last_became_peered":"2024-03-16T10:11:40.033873+0000","last_unstale":"2024-03-16T10:11:42.934753+0000","last_undegraded":"2024-03-16T10:11:42.934753+0000","last_fullsized":"2024-03-16T10:11:42.934753+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:34:00.943144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.025911+0000","last_change":"2024-03-16T10:11:39.042466+0000","last_active":"2024-03-16T10:11:41.025911+0000","last_peered":"2024-03-16T10:11:41.025911+0000","last_clean":"2024-03-16T10:11:41.025911+0000","last_became_active":"2024-03-16T10:11:39.042100+0000","last_became_peered":"2024-03-16T10:11:39.042100+0000","last_unstale":"2024-03-16T10:11:41.025911+0000","last_undegraded":"2024-03-16T10:11:41.025911+0000","last_fullsized":"2024-03-16T10:11:41.025911+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:39.788886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935017+0000","last_change":"2024-03-16T10:11:40.031195+0000","last_active":"2024-03-16T10:11:42.935017+0000","last_peered":"2024-03-16T10:11:42.935017+0000","last_clean":"2024-03-16T10:11:42.935017+0000","last_became_active":"2024-03-16T10:11:40.030881+0000","last_became_peered":"2024-03-16T10:11:40.030881+0000","last_unstale":"2024-03-16T10:11:42.935017+0000","last_undegraded":"2024-03-16T10:11:42.935017+0000","last_fullsized":"2024-03-16T10:11:42.935017+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:02.965529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.029700+0000","last_change":"2024-03-16T10:11:39.044909+0000","last_active":"2024-03-16T10:11:41.029700+0000","last_peered":"2024-03-16T10:11:41.029700+0000","last_clean":"2024-03-16T10:11:41.029700+0000","last_became_active":"2024-03-16T10:11:39.044151+0000","last_became_peered":"2024-03-16T10:11:39.044151+0000","last_unstale":"2024-03-16T10:11:41.029700+0000","last_undegraded":"2024-03-16T10:11:41.029700+0000","last_fullsized":"2024-03-16T10:11:41.029700+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.934791+0000","last_change":"2024-03-16T10:11:40.031271+0000","last_active":"2024-03-16T10:11:42.934791+0000","last_peered":"2024-03-16T10:11:42.934791+0000","last_clean":"2024-03-16T10:11:42.934791+0000","last_became_active":"2024-03-16T10:11:40.030860+0000","last_became_peered":"2024-03-16T10:11:40.030860+0000","last_unstale":"2024-03-16T10:11:42.934791+0000","last_undegraded":"2024-03-16T10:11:42.934791+0000","last_fullsized":"2024-03-16T10:11:42.934791+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:12:13.892946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.028319+0000","last_change":"2024-03-16T10:11:39.059806+0000","last_active":"2024-03-16T10:11:41.028319+0000","last_peered":"2024-03-16T10:11:41.028319+0000","last_clean":"2024-03-16T10:11:41.028319+0000","last_became_active":"2024-03-16T10:11:39.053858+0000","last_became_peered":"2024-03-16T10:11:39.053858+0000","last_unstale":"2024-03-16T10:11:41.028319+0000","last_undegraded":"2024-03-16T10:11:41.028319+0000","last_fullsized":"2024-03-16T10:11:41.028319+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:15:51.282542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.264130+0000","last_change":"2024-03-16T10:11:40.074060+0000","last_active":"2024-03-16T10:11:41.264130+0000","last_peered":"2024-03-16T10:11:41.264130+0000","last_clean":"2024-03-16T10:11:41.264130+0000","last_became_active":"2024-03-16T10:11:40.073357+0000","last_became_peered":"2024-03-16T10:11:40.073357+0000","last_unstale":"2024-03-16T10:11:41.264130+0000","last_undegraded":"2024-03-16T10:11:41.264130+0000","last_fullsized":"2024-03-16T10:11:41.264130+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:56:04.173993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.937336+0000","last_change":"2024-03-16T10:11:39.051315+0000","last_active":"2024-03-16T10:11:42.937336+0000","last_peered":"2024-03-16T10:11:42.937336+0000","last_clean":"2024-03-16T10:11:42.937336+0000","last_became_active":"2024-03-16T10:11:39.050550+0000","last_became_peered":"2024-03-16T10:11:39.050550+0000","last_unstale":"2024-03-16T10:11:42.937336+0000","last_undegraded":"2024-03-16T10:11:42.937336+0000","last_fullsized":"2024-03-16T10:11:42.937336+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:11:55.739859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935114+0000","last_change":"2024-03-16T10:11:40.061514+0000","last_active":"2024-03-16T10:11:42.935114+0000","last_peered":"2024-03-16T10:11:42.935114+0000","last_clean":"2024-03-16T10:11:42.935114+0000","last_became_active":"2024-03-16T10:11:40.061174+0000","last_became_peered":"2024-03-16T10:11:40.061174+0000","last_unstale":"2024-03-16T10:11:42.935114+0000","last_undegraded":"2024-03-16T10:11:42.935114+0000","last_fullsized":"2024-03-16T10:11:42.935114+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:40.792776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.029058+0000","last_change":"2024-03-16T10:11:39.060338+0000","last_active":"2024-03-16T10:11:42.029058+0000","last_peered":"2024-03-16T10:11:42.029058+0000","last_clean":"2024-03-16T10:11:42.029058+0000","last_became_active":"2024-03-16T10:11:39.060040+0000","last_became_peered":"2024-03-16T10:11:39.060040+0000","last_unstale":"2024-03-16T10:11:42.029058+0000","last_undegraded":"2024-03-16T10:11:42.029058+0000","last_fullsized":"2024-03-16T10:11:42.029058+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:12:39.788886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.938482+0000","last_change":"2024-03-16T10:11:40.075328+0000","last_active":"2024-03-16T10:11:42.938482+0000","last_peered":"2024-03-16T10:11:42.938482+0000","last_clean":"2024-03-16T10:11:42.938482+0000","last_became_active":"2024-03-16T10:11:40.074731+0000","last_became_peered":"2024-03-16T10:11:40.074731+0000","last_unstale":"2024-03-16T10:11:42.938482+0000","last_undegraded":"2024-03-16T10:11:42.938482+0000","last_fullsized":"2024-03-16T10:11:42.938482+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:20:01.599220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"30'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.026632+0000","last_change":"2024-03-16T10:11:39.042670+0000","last_active":"2024-03-16T10:11:41.026632+0000","last_peered":"2024-03-16T10:11:41.026632+0000","last_clean":"2024-03-16T10:11:41.026632+0000","last_became_active":"2024-03-16T10:11:39.042250+0000","last_became_peered":"2024-03-16T10:11:39.042250+0000","last_unstale":"2024-03-16T10:11:41.026632+0000","last_undegraded":"2024-03-16T10:11:41.026632+0000","last_fullsized":"2024-03-16T10:11:41.026632+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.025857+0000","last_change":"2024-03-16T10:11:40.035336+0000","last_active":"2024-03-16T10:11:41.025857+0000","last_peered":"2024-03-16T10:11:41.025857+0000","last_clean":"2024-03-16T10:11:41.025857+0000","last_became_active":"2024-03-16T10:11:40.035053+0000","last_became_peered":"2024-03-16T10:11:40.035053+0000","last_unstale":"2024-03-16T10:11:41.025857+0000","last_undegraded":"2024-03-16T10:11:41.025857+0000","last_fullsized":"2024-03-16T10:11:41.025857+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:11:56.743750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.028384+0000","last_change":"2024-03-16T10:11:39.034786+0000","last_active":"2024-03-16T10:11:42.028384+0000","last_peered":"2024-03-16T10:11:42.028384+0000","last_clean":"2024-03-16T10:11:42.028384+0000","last_became_active":"2024-03-16T10:11:39.034431+0000","last_became_peered":"2024-03-16T10:11:39.034431+0000","last_unstale":"2024-03-16T10:11:42.028384+0000","last_undegraded":"2024-03-16T10:11:42.028384+0000","last_fullsized":"2024-03-16T10:11:42.028384+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T19:34:20.376645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.029450+0000","last_change":"2024-03-16T10:11:40.067068+0000","last_active":"2024-03-16T10:11:41.029450+0000","last_peered":"2024-03-16T10:11:41.029450+0000","last_clean":"2024-03-16T10:11:41.029450+0000","last_became_active":"2024-03-16T10:11:40.066240+0000","last_became_peered":"2024-03-16T10:11:40.066240+0000","last_unstale":"2024-03-16T10:11:41.029450+0000","last_undegraded":"2024-03-16T10:11:41.029450+0000","last_fullsized":"2024-03-16T10:11:41.029450+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:45:21.820367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.026144+0000","last_change":"2024-03-16T10:11:39.054972+0000","last_active":"2024-03-16T10:11:41.026144+0000","last_peered":"2024-03-16T10:11:41.026144+0000","last_clean":"2024-03-16T10:11:41.026144+0000","last_became_active":"2024-03-16T10:11:39.054554+0000","last_became_peered":"2024-03-16T10:11:39.054554+0000","last_unstale":"2024-03-16T10:11:41.026144+0000","last_undegraded":"2024-03-16T10:11:41.026144+0000","last_fullsized":"2024-03-16T10:11:41.026144+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T15:59:48.922456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.025999+0000","last_change":"2024-03-16T10:11:40.060171+0000","last_active":"2024-03-16T10:11:41.025999+0000","last_peered":"2024-03-16T10:11:41.025999+0000","last_clean":"2024-03-16T10:11:41.025999+0000","last_became_active":"2024-03-16T10:11:40.059815+0000","last_became_peered":"2024-03-16T10:11:40.059815+0000","last_unstale":"2024-03-16T10:11:41.025999+0000","last_undegraded":"2024-03-16T10:11:41.025999+0000","last_fullsized":"2024-03-16T10:11:41.025999+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T18:49:19.819589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.028095+0000","last_change":"2024-03-16T10:11:39.055099+0000","last_active":"2024-03-16T10:11:42.028095+0000","last_peered":"2024-03-16T10:11:42.028095+0000","last_clean":"2024-03-16T10:11:42.028095+0000","last_became_active":"2024-03-16T10:11:39.054743+0000","last_became_peered":"2024-03-16T10:11:39.054743+0000","last_unstale":"2024-03-16T10:11:42.028095+0000","last_undegraded":"2024-03-16T10:11:42.028095+0000","last_fullsized":"2024-03-16T10:11:42.028095+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:14:01.961638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935534+0000","last_change":"2024-03-16T10:11:40.045840+0000","last_active":"2024-03-16T10:11:42.935534+0000","last_peered":"2024-03-16T10:11:42.935534+0000","last_clean":"2024-03-16T10:11:42.935534+0000","last_became_active":"2024-03-16T10:11:40.040556+0000","last_became_peered":"2024-03-16T10:11:40.040556+0000","last_unstale":"2024-03-16T10:11:42.935534+0000","last_undegraded":"2024-03-16T10:11:42.935534+0000","last_fullsized":"2024-03-16T10:11:42.935534+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:38:20.425271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.028579+0000","last_change":"2024-03-16T10:11:39.053883+0000","last_active":"2024-03-16T10:11:41.028579+0000","last_peered":"2024-03-16T10:11:41.028579+0000","last_clean":"2024-03-16T10:11:41.028579+0000","last_became_active":"2024-03-16T10:11:39.052424+0000","last_became_peered":"2024-03-16T10:11:39.052424+0000","last_unstale":"2024-03-16T10:11:41.028579+0000","last_undegraded":"2024-03-16T10:11:41.028579+0000","last_fullsized":"2024-03-16T10:11:41.028579+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T13:35:59.698889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.028059+0000","last_change":"2024-03-16T10:11:40.049061+0000","last_active":"2024-03-16T10:11:41.028059+0000","last_peered":"2024-03-16T10:11:41.028059+0000","last_clean":"2024-03-16T10:11:41.028059+0000","last_became_active":"2024-03-16T10:11:40.048340+0000","last_became_peered":"2024-03-16T10:11:40.048340+0000","last_unstale":"2024-03-16T10:11:41.028059+0000","last_undegraded":"2024-03-16T10:11:41.028059+0000","last_fullsized":"2024-03-16T10:11:41.028059+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T12:45:21.820367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:42.935648+0000","last_change":"2024-03-16T10:11:39.049474+0000","last_active":"2024-03-16T10:11:42.935648+0000","last_peered":"2024-03-16T10:11:42.935648+0000","last_clean":"2024-03-16T10:11:42.935648+0000","last_became_active":"2024-03-16T10:11:39.049180+0000","last_became_peered":"2024-03-16T10:11:39.049180+0000","last_unstale":"2024-03-16T10:11:42.935648+0000","last_undegraded":"2024-03-16T10:11:42.935648+0000","last_fullsized":"2024-03-16T10:11:42.935648+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T14:34:23.568982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.264482+0000","last_change":"2024-03-16T10:11:40.060808+0000","last_active":"2024-03-16T10:11:41.264482+0000","last_peered":"2024-03-16T10:11:41.264482+0000","last_clean":"2024-03-16T10:11:41.264482+0000","last_became_active":"2024-03-16T10:11:40.055371+0000","last_became_peered":"2024-03-16T10:11:40.055371+0000","last_unstale":"2024-03-16T10:11:41.264482+0000","last_undegraded":"2024-03-16T10:11:41.264482+0000","last_fullsized":"2024-03-16T10:11:41.264482+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:39.007282+0000","last_clean_scrub_stamp":"2024-03-16T10:11:39.007282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T10:38:20.425271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.027778+0000","last_change":"2024-03-16T10:11:39.055688+0000","last_active":"2024-03-16T10:11:41.027778+0000","last_peered":"2024-03-16T10:11:41.027778+0000","last_clean":"2024-03-16T10:11:41.027778+0000","last_became_active":"2024-03-16T10:11:39.054833+0000","last_became_peered":"2024-03-16T10:11:39.054833+0000","last_unstale":"2024-03-16T10:11:41.027778+0000","last_undegraded":"2024-03-16T10:11:41.027778+0000","last_fullsized":"2024-03-16T10:11:41.027778+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:11:38.003391+0000","last_clean_scrub_stamp":"2024-03-16T10:11:38.003391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T16:13:57.217211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T10:11:41.265100+0000","last_change":"2024-03-16T10:10:48.248686+0000","last_active":"2024-03-16T10:11:41.265100+0000","last_peered":"2024-03-16T10:11:41.265100+0000","last_clean":"2024-03-16T10:11:41.265100+0000","last_became_active":"2024-03-16T10:10:48.243054+0000","last_became_peered":"2024-03-16T10:10:48.243054+0000","last_unstale":"2024-03-16T10:11:41.265100+0000","last_undegraded":"2024-03-16T10:11:41.265100+0000","last_fullsized":"2024-03-16T10:11:41.265100+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_clean_scrub_stamp":"2024-03-16T10:10:46.517938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:42.906473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":933888,"data_stored":719141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":129299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":129355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":128787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":130317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":339968,"data_stored":128753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":130317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":929792,"data_stored":719051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:11:48.083 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-16T10:11:48.084 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-16T10:11:48.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.272+0000 7f76ba09a640 1 Processor -- start 2024-03-16T10:11:48.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.272+0000 7f76ba09a640 1 -- start start 2024-03-16T10:11:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76ba09a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76b40bd410 0x7f76b4141140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76ba09a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b40bd7f0 con 0x7f76b4141680 2024-03-16T10:11:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76ba09a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76b40b5780 con 0x7f76b414b1e0 2024-03-16T10:11:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76ba09a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f76b40b5af0 con 0x7f76b40bd410 2024-03-16T10:11:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76abfff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f76b4141680 0x7f76b414aad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48516/0 (socket says 172.21.15.17:48516) 2024-03-16T10:11:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76abfff640 1 -- 172.21.15.17:0/2549443035 learned_addr learned my addr 172.21.15.17:0/2549443035 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b37fe640 1 --2- 172.21.15.17:0/2549443035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76b40bd410 0x7f76b4141140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 -- 172.21.15.17:0/2549443035 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 194063232 0 0) 0x7f76b40bd7f0 con 0x7f76b4141680 2024-03-16T10:11:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 -- 172.21.15.17:0/2549443035 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76a0003530 con 0x7f76b4141680 2024-03-16T10:11:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 -- 172.21.15.17:0/2549443035 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2528307491 0 0) 0x7f76a0003530 con 0x7f76b4141680 2024-03-16T10:11:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 -- 172.21.15.17:0/2549443035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76b40bd410 msgr2=0x7f76b4141140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 --2- 172.21.15.17:0/2549443035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76b40bd410 0x7f76b4141140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 -- 172.21.15.17:0/2549443035 >> v1:172.21.15.174:6789/0 conn(0x7f76b414b1e0 legacy=0x7f76b4151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 -- 172.21.15.17:0/2549443035 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b40b4d90 con 0x7f76b4141680 2024-03-16T10:11:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 -- 172.21.15.17:0/2549443035 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f769c001680 con 0x7f76b4141680 2024-03-16T10:11:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 -- 172.21.15.17:0/2549443035 >> v1:172.21.15.17:6789/0 conn(0x7f76b4141680 legacy=0x7f76b414aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 --2- 172.21.15.17:0/2549443035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76a00039f0 0x7f76a0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 --2- 172.21.15.17:0/2549443035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0004810 0x7f76a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 --2- 172.21.15.17:0/2549443035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76b40bd410 0x7f76a0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 -- 172.21.15.17:0/2549443035 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f76b40b5af0 con 0x7f76a00039f0 2024-03-16T10:11:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 -- 172.21.15.17:0/2549443035 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76a000bb00 con 0x7f76a0004810 2024-03-16T10:11:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 -- 172.21.15.17:0/2549443035 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f76a000bd00 con 0x7f76b40bd410 2024-03-16T10:11:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b37fe640 1 --2- 172.21.15.17:0/2549443035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76a00039f0 0x7f76a0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76abfff640 1 --2- 172.21.15.17:0/2549443035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0004810 0x7f76a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b3fff640 1 --2- 172.21.15.17:0/2549443035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76b40bd410 0x7f76a0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b37fe640 1 -- 172.21.15.17:0/2549443035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76b40bd410 msgr2=0x7f76a0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b37fe640 1 --2- 172.21.15.17:0/2549443035 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76b40bd410 0x7f76a0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b37fe640 1 -- 172.21.15.17:0/2549443035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0004810 msgr2=0x7f76a0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b37fe640 1 --2- 172.21.15.17:0/2549443035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76a0004810 0x7f76a0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b37fe640 1 -- 172.21.15.17:0/2549443035 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a0003530 con 0x7f76a00039f0 2024-03-16T10:11:48.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b37fe640 1 --2- 172.21.15.17:0/2549443035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76a00039f0 0x7f76a0003e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009ac0 tx=0x7f76a401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864932f1c63ca398 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:48.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 -- 172.21.15.17:0/2549443035 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a4009690 con 0x7f76a00039f0 2024-03-16T10:11:48.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 -- 172.21.15.17:0/2549443035 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a40a6430 con 0x7f76a00039f0 2024-03-16T10:11:48.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.276+0000 7f76b2ffd640 1 -- 172.21.15.17:0/2549443035 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a40a6800 con 0x7f76a00039f0 2024-03-16T10:11:48.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 -- 172.21.15.17:0/2549443035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76a00039f0 msgr2=0x7f76a0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:48.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 --2- 172.21.15.17:0/2549443035 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76a00039f0 0x7f76a0003e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009ac0 tx=0x7f76a401ca50 comp rx=0 tx=0).stop 2024-03-16T10:11:48.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76b3fff640 1 -- 172.21.15.17:0/2549443035 reap_dead start 2024-03-16T10:11:48.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 -- 172.21.15.17:0/2549443035 shutdown_connections 2024-03-16T10:11:48.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 -- 172.21.15.17:0/2549443035 >> 172.21.15.17:0/2549443035 conn(0x7f76b4072dc0 msgr2=0x7f76b40731d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 -- 172.21.15.17:0/2549443035 shutdown_connections 2024-03-16T10:11:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 -- 172.21.15.17:0/2549443035 wait complete. 2024-03-16T10:11:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 Processor -- start 2024-03-16T10:11:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 -- start start 2024-03-16T10:11:48.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76b40bbe00 0x7f76b41dff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76b40bc1e0 0x7f76b41e0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76b4149ca0 0x7f76b414a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f76b4153d60 con 0x7f76b40bbe00 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76b4153be0 con 0x7f76b40bc1e0 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f76b4153ee0 con 0x7f76b4149ca0 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76b37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76b40bbe00 0x7f76b41dff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76b37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76b40bbe00 0x7f76b41dff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52564/0 (socket says 172.21.15.17:52564) 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76b37fe640 1 -- 172.21.15.17:0/2799114789 learned_addr learned my addr 172.21.15.17:0/2799114789 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76abfff640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76b40bc1e0 0x7f76b41e0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76b37fe640 1 -- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76b4149ca0 msgr2=0x7f76b414a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76b3fff640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76b4149ca0 0x7f76b414a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76b37fe640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76b4149ca0 0x7f76b414a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76b37fe640 1 -- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76b40bc1e0 msgr2=0x7f76b41e0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76b37fe640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76b40bc1e0 0x7f76b41e0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76b37fe640 1 -- 172.21.15.17:0/2799114789 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b413c100 con 0x7f76b40bbe00 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76abfff640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76b40bc1e0 0x7f76b41e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76b37fe640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76b40bbe00 0x7f76b41dff50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009ac0 tx=0x7f76a40197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76b0ff9640 1 -- 172.21.15.17:0/2799114789 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a40a6cc0 con 0x7f76b40bbe00 2024-03-16T10:11:48.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76b0ff9640 1 -- 172.21.15.17:0/2799114789 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76a40ba440 con 0x7f76b40bbe00 2024-03-16T10:11:48.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 -- 172.21.15.17:0/2799114789 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b41e3a70 con 0x7f76b40bbe00 2024-03-16T10:11:48.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76b0ff9640 1 -- 172.21.15.17:0/2799114789 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a40ba830 con 0x7f76b40bbe00 2024-03-16T10:11:48.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.280+0000 7f76ba09a640 1 -- 172.21.15.17:0/2799114789 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b41e4030 con 0x7f76b40bbe00 2024-03-16T10:11:48.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.284+0000 7f76ba09a640 1 -- 172.21.15.17:0/2799114789 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b4068360 con 0x7f76b40bbe00 2024-03-16T10:11:48.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.284+0000 7f76b0ff9640 1 -- 172.21.15.17:0/2799114789 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f76a40a7410 con 0x7f76b40bbe00 2024-03-16T10:11:48.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.284+0000 7f76b0ff9640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7694067690 0x7f7694069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.284+0000 7f76b0ff9640 1 -- 172.21.15.17:0/2799114789 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f76a411f7a0 con 0x7f76b40bbe00 2024-03-16T10:11:48.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.284+0000 7f76abfff640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7694067690 0x7f7694069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.284+0000 7f76abfff640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7694067690 0x7f7694069b50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f769c001060 tx=0x7f769c001d80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:48.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.288+0000 7f76b0ff9640 1 -- 172.21.15.17:0/2799114789 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f76a40ac020 con 0x7f76b40bbe00 2024-03-16T10:11:48.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.516+0000 7f76ba09a640 1 -- 172.21.15.17:0/2799114789 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f76b4144d50 con 0x7f76b40bbe00 2024-03-16T10:11:48.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.520+0000 7f76b0ff9640 1 -- 172.21.15.17:0/2799114789 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f76a40f4e20 con 0x7f76b40bbe00 2024-03-16T10:11:48.525 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-03-16T10:11:48.525 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD6b/Vl9GM/ARAAE2ObwC6ro6fuTOmqZ2GNzg== 2024-03-16T10:11:48.525 INFO:teuthology.orchestra.run.smithi017.stdout: caps mds = "allow" 2024-03-16T10:11:48.525 INFO:teuthology.orchestra.run.smithi017.stdout: caps mon = "allow r" 2024-03-16T10:11:48.525 INFO:teuthology.orchestra.run.smithi017.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-16T10:11:48.525 INFO:teuthology.orchestra.run.smithi017.stderr:updated caps for client.0 2024-03-16T10:11:48.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.520+0000 7f76ba09a640 1 -- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7694067690 msgr2=0x7f7694069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:48.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.520+0000 7f76ba09a640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7694067690 0x7f7694069b50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f769c001060 tx=0x7f769c001d80 comp rx=0 tx=0).stop 2024-03-16T10:11:48.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.520+0000 7f76ba09a640 1 -- 172.21.15.17:0/2799114789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76b40bbe00 msgr2=0x7f76b41dff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:48.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.520+0000 7f76ba09a640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76b40bbe00 0x7f76b41dff50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009ac0 tx=0x7f76a40197f0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.520+0000 7f76ba09a640 1 -- 172.21.15.17:0/2799114789 shutdown_connections 2024-03-16T10:11:48.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.520+0000 7f76ba09a640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7694067690 0x7f7694069b50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.520+0000 7f76ba09a640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f76b4149ca0 0x7f76b414a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.520+0000 7f76ba09a640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76b40bc1e0 0x7f76b41e0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.520+0000 7f76ba09a640 1 --2- 172.21.15.17:0/2799114789 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f76b40bbe00 0x7f76b41dff50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.520+0000 7f76ba09a640 1 -- 172.21.15.17:0/2799114789 >> 172.21.15.17:0/2799114789 conn(0x7f76b4072dc0 msgr2=0x7f76b40b8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:48.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.520+0000 7f76ba09a640 1 -- 172.21.15.17:0/2799114789 shutdown_connections 2024-03-16T10:11:48.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.520+0000 7f76ba09a640 1 -- 172.21.15.17:0/2799114789 wait complete. 2024-03-16T10:11:48.544 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:11:48.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.648+0000 7f8759d69640 1 Processor -- start 2024-03-16T10:11:48.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.648+0000 7f8759d69640 1 -- start start 2024-03-16T10:11:48.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8759d69640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f87540b7650 0x7f87541412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8759d69640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8754141c30 con 0x7f87540b4650 2024-03-16T10:11:48.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8759d69640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87540b4a30 con 0x7f8754141820 2024-03-16T10:11:48.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8759d69640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f875414abe0 con 0x7f87540b7650 2024-03-16T10:11:48.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f87537fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f87540b4650 0x7f875414b3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:48526/0 (socket says 172.21.15.17:48526) 2024-03-16T10:11:48.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f87537fe640 1 -- 172.21.15.17:0/2698621820 learned_addr learned my addr 172.21.15.17:0/2698621820 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:48.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f874bfff640 1 --2- 172.21.15.17:0/2698621820 >> v2:172.21.15.174:3300/0 conn(0x7f87540b7650 0x7f87541412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 -- 172.21.15.17:0/2698621820 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1321416765 0 0) 0x7f8754141c30 con 0x7f87540b4650 2024-03-16T10:11:48.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 -- 172.21.15.17:0/2698621820 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8740003530 con 0x7f87540b4650 2024-03-16T10:11:48.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 -- 172.21.15.17:0/2698621820 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3382813282 0 0) 0x7f8740003530 con 0x7f87540b4650 2024-03-16T10:11:48.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 -- 172.21.15.17:0/2698621820 >> v2:172.21.15.174:3300/0 conn(0x7f87540b7650 msgr2=0x7f87541412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:48.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 --2- 172.21.15.17:0/2698621820 >> v2:172.21.15.174:3300/0 conn(0x7f87540b7650 0x7f87541412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 -- 172.21.15.17:0/2698621820 >> v1:172.21.15.174:6789/0 conn(0x7f8754141820 legacy=0x7f8754151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:48.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 -- 172.21.15.17:0/2698621820 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875414aec0 con 0x7f87540b4650 2024-03-16T10:11:48.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 -- 172.21.15.17:0/2698621820 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f8744001680 con 0x7f87540b4650 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 -- 172.21.15.17:0/2698621820 >> v1:172.21.15.17:6789/0 conn(0x7f87540b4650 legacy=0x7f875414b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 --2- 172.21.15.17:0/2698621820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87400039c0 0x7f8740003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 --2- 172.21.15.17:0/2698621820 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8740004810 0x7f8740004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 --2- 172.21.15.17:0/2698621820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87400051d0 0x7f874000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 -- 172.21.15.17:0/2698621820 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f875414abe0 con 0x7f87400039c0 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 -- 172.21.15.17:0/2698621820 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f874000c170 con 0x7f87400051d0 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8752ffd640 1 -- 172.21.15.17:0/2698621820 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f874000c370 con 0x7f8740004810 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f87537fe640 1 --2- 172.21.15.17:0/2698621820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87400039c0 0x7f8740003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8753fff640 1 --2- 172.21.15.17:0/2698621820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87400051d0 0x7f874000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f87537fe640 1 -- 172.21.15.17:0/2698621820 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8740004810 msgr2=0x7f8740004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f87537fe640 1 --2- 172.21.15.17:0/2698621820 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8740004810 0x7f8740004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f87537fe640 1 -- 172.21.15.17:0/2698621820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87400051d0 msgr2=0x7f874000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f87537fe640 1 --2- 172.21.15.17:0/2698621820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87400051d0 0x7f874000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f87537fe640 1 -- 172.21.15.17:0/2698621820 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8740003530 con 0x7f87400039c0 2024-03-16T10:11:48.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.652+0000 7f8753fff640 1 -- 172.21.15.17:0/2698621820 reap_dead start 2024-03-16T10:11:48.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f87537fe640 1 --2- 172.21.15.17:0/2698621820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87400039c0 0x7f8740003de0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f874400c240 tx=0x7f874401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604922ea420380a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:48.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8752ffd640 1 -- 172.21.15.17:0/2698621820 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744007d20 con 0x7f87400039c0 2024-03-16T10:11:48.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8752ffd640 1 -- 172.21.15.17:0/2698621820 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87440a9070 con 0x7f87400039c0 2024-03-16T10:11:48.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8752ffd640 1 -- 172.21.15.17:0/2698621820 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744005da0 con 0x7f87400039c0 2024-03-16T10:11:48.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 -- 172.21.15.17:0/2698621820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87400039c0 msgr2=0x7f8740003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:48.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 --2- 172.21.15.17:0/2698621820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87400039c0 0x7f8740003de0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f874400c240 tx=0x7f874401cc80 comp rx=0 tx=0).stop 2024-03-16T10:11:48.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 -- 172.21.15.17:0/2698621820 shutdown_connections 2024-03-16T10:11:48.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 --2- 172.21.15.17:0/2698621820 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87400039c0 0x7f8740003de0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 -- 172.21.15.17:0/2698621820 >> 172.21.15.17:0/2698621820 conn(0x7f87540bccd0 msgr2=0x7f87540bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:48.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 -- 172.21.15.17:0/2698621820 shutdown_connections 2024-03-16T10:11:48.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 -- 172.21.15.17:0/2698621820 wait complete. 2024-03-16T10:11:48.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 Processor -- start 2024-03-16T10:11:48.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 -- start start 2024-03-16T10:11:48.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87400039c0 0x7f87541d5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87541e0170 0x7f875414a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8754144530 0x7f875413cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f87540b94a0 con 0x7f87400039c0 2024-03-16T10:11:48.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87540b9320 con 0x7f87541e0170 2024-03-16T10:11:48.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f87540b59f0 con 0x7f8754144530 2024-03-16T10:11:48.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f87537fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87400039c0 0x7f87541d5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f87537fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87400039c0 0x7f87541d5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52590/0 (socket says 172.21.15.17:52590) 2024-03-16T10:11:48.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f87537fe640 1 -- 172.21.15.17:0/585052539 learned_addr learned my addr 172.21.15.17:0/585052539 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:48.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8753fff640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8754144530 0x7f875413cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f874bfff640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87541e0170 0x7f875414a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f87537fe640 1 -- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8754144530 msgr2=0x7f875413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:48.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f87537fe640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8754144530 0x7f875413cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f87537fe640 1 -- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87541e0170 msgr2=0x7f875414a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:48.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f87537fe640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87541e0170 0x7f875414a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:48.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f87537fe640 1 -- 172.21.15.17:0/585052539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875414a940 con 0x7f87400039c0 2024-03-16T10:11:48.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f874bfff640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87541e0170 0x7f875414a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:48.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f87537fe640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87400039c0 0x7f87541d5fa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f874401cab0 tx=0x7f8740005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:48.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8750ff9640 1 -- 172.21.15.17:0/585052539 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744007d20 con 0x7f87400039c0 2024-03-16T10:11:48.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8750ff9640 1 -- 172.21.15.17:0/585052539 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8744000ee0 con 0x7f87400039c0 2024-03-16T10:11:48.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8750ff9640 1 -- 172.21.15.17:0/585052539 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8744001480 con 0x7f87400039c0 2024-03-16T10:11:48.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 -- 172.21.15.17:0/585052539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8754149f00 con 0x7f87400039c0 2024-03-16T10:11:48.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.656+0000 7f8759d69640 1 -- 172.21.15.17:0/585052539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8754144d80 con 0x7f87400039c0 2024-03-16T10:11:48.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.660+0000 7f8759d69640 1 -- 172.21.15.17:0/585052539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8754068360 con 0x7f87400039c0 2024-03-16T10:11:48.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.660+0000 7f8750ff9640 1 -- 172.21.15.17:0/585052539 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8744007180 con 0x7f87400039c0 2024-03-16T10:11:48.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.660+0000 7f8750ff9640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8734067660 0x7f8734069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:48.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.660+0000 7f8750ff9640 1 -- 172.21.15.17:0/585052539 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f874411f060 con 0x7f87400039c0 2024-03-16T10:11:48.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.660+0000 7f874bfff640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8734067660 0x7f8734069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:48.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.660+0000 7f874bfff640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8734067660 0x7f8734069b20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f87540b3720 tx=0x7f873c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:48.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:48.664+0000 7f8750ff9640 1 -- 172.21.15.17:0/585052539 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8744016990 con 0x7f87400039c0 2024-03-16T10:11:49.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.072+0000 7f8759d69640 1 -- 172.21.15.17:0/585052539 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87540b90c0 con 0x7f87400039c0 2024-03-16T10:11:49.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.072+0000 7f8750ff9640 1 -- 172.21.15.17:0/585052539 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+3531 (secure 0 0 0) 0x7f87440efce0 con 0x7f87400039c0 2024-03-16T10:11:49.077 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:49.077 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4738,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1674220578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1674220578},{"type":"v1","addr":"172.21.15.174:6835","nonce":1674220578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:11:39.392755+0000","modified":"2024-03-16T10:11:41.419952+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6833/578943207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":578943207},{"type":"v1","addr":"172.21.15.17:6833","nonce":578943207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4953,"qdb_cluster":[4953]},"id":2}]} 2024-03-16T10:11:49.078 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 15 2024-03-16T10:11:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.076+0000 7f8759d69640 1 -- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8734067660 msgr2=0x7f8734069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.076+0000 7f8759d69640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8734067660 0x7f8734069b20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f87540b3720 tx=0x7f873c008040 comp rx=0 tx=0).stop 2024-03-16T10:11:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.076+0000 7f8759d69640 1 -- 172.21.15.17:0/585052539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87400039c0 msgr2=0x7f87541d5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.076+0000 7f8759d69640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87400039c0 0x7f87541d5fa0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f874401cab0 tx=0x7f8740005840 comp rx=0 tx=0).stop 2024-03-16T10:11:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.076+0000 7f8759d69640 1 -- 172.21.15.17:0/585052539 shutdown_connections 2024-03-16T10:11:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.076+0000 7f8759d69640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8734067660 0x7f8734069b20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.076+0000 7f8759d69640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8754144530 0x7f875413cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.076+0000 7f8759d69640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87541e0170 0x7f875414a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.076+0000 7f8759d69640 1 --2- 172.21.15.17:0/585052539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87400039c0 0x7f87541d5fa0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.076+0000 7f8759d69640 1 -- 172.21.15.17:0/585052539 >> 172.21.15.17:0/585052539 conn(0x7f87540bccd0 msgr2=0x7f875413f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.076+0000 7f8759d69640 1 -- 172.21.15.17:0/585052539 shutdown_connections 2024-03-16T10:11:49.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.076+0000 7f8759d69640 1 -- 172.21.15.17:0/585052539 wait complete. 2024-03-16T10:11:49.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-16T10:11:39.392755+0000', 'modified': '2024-03-16T10:11:41.419952+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4953}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4953': {'gid': 4953, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.17:6833/578943207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.17:6832', 'nonce': 578943207}, {'type': 'v1', 'addr': '172.21.15.17:6833', 'nonce': 578943207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4953, 'qdb_cluster': [4953]} 2024-03-16T10:11:49.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-16T10:11:49.298 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-16T10:11:49.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2f2ec5640 1 Processor -- start 2024-03-16T10:11:49.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2f2ec5640 1 -- start start 2024-03-16T10:11:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2f2ec5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2ec0b7650 0x7fb2ec1412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2f2ec5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb2ec141820 0x7fb2ec151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2f2ec5640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2ec141c50 con 0x7fb2ec0b4650 2024-03-16T10:11:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2f2ec5640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb2ec0b4a30 con 0x7fb2ec141820 2024-03-16T10:11:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2f2ec5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb2ec14abe0 con 0x7fb2ec0b7650 2024-03-16T10:11:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2f0c61640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb2ec0b4650 0x7fb2ec14b3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42858/0 (socket says 172.21.15.17:42858) 2024-03-16T10:11:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2f1462640 1 --2- 172.21.15.17:0/462894333 >> v2:172.21.15.17:3300/0 conn(0x7fb2ec141820 0x7fb2ec151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2f0c61640 1 -- 172.21.15.17:0/462894333 learned_addr learned my addr 172.21.15.17:0/462894333 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2ebfff640 1 --2- 172.21.15.17:0/462894333 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2ec0b7650 0x7fb2ec1412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2f1462640 1 -- 172.21.15.17:0/462894333 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2ec0b7650 msgr2=0x7fb2ec1412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2f1462640 1 --2- 172.21.15.17:0/462894333 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2ec0b7650 0x7fb2ec1412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2f1462640 1 -- 172.21.15.17:0/462894333 >> v1:172.21.15.17:6789/0 conn(0x7fb2ec0b4650 legacy=0x7fb2ec14b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2eb7fe640 1 -- 172.21.15.17:0/462894333 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3401132001 0 0) 0x7fb2ec141c50 con 0x7fb2ec0b4650 2024-03-16T10:11:49.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.420+0000 7fb2f1462640 1 -- 172.21.15.17:0/462894333 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ec14aec0 con 0x7fb2ec141820 2024-03-16T10:11:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.424+0000 7fb2f1462640 1 --2- 172.21.15.17:0/462894333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2ec141820 0x7fb2ec151dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009d90 tx=0x7fb2e401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad515374b75f2e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.424+0000 7fb2eb7fe640 1 -- 172.21.15.17:0/462894333 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40b2070 con 0x7fb2ec141820 2024-03-16T10:11:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.424+0000 7fb2eb7fe640 1 -- 172.21.15.17:0/462894333 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e40a57a0 con 0x7fb2ec141820 2024-03-16T10:11:49.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.424+0000 7fb2eb7fe640 1 -- 172.21.15.17:0/462894333 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40a5b70 con 0x7fb2ec141820 2024-03-16T10:11:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 -- 172.21.15.17:0/462894333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2ec141820 msgr2=0x7fb2ec151dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 --2- 172.21.15.17:0/462894333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2ec141820 0x7fb2ec151dd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009d90 tx=0x7fb2e401c8e0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 -- 172.21.15.17:0/462894333 shutdown_connections 2024-03-16T10:11:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 --2- 172.21.15.17:0/462894333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2ec141820 0x7fb2ec151dd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 --2- 172.21.15.17:0/462894333 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2ec0b7650 0x7fb2ec1412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 -- 172.21.15.17:0/462894333 >> 172.21.15.17:0/462894333 conn(0x7fb2ec0bccd0 msgr2=0x7fb2ec0bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 -- 172.21.15.17:0/462894333 shutdown_connections 2024-03-16T10:11:49.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 -- 172.21.15.17:0/462894333 wait complete. 2024-03-16T10:11:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 Processor -- start 2024-03-16T10:11:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 -- start start 2024-03-16T10:11:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2ec0b4650 0x7fb2ec1395a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2ec0b7650 0x7fb2ec143a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2ec141820 0x7fb2ec1e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec0b5b70 con 0x7fb2ec0b7650 2024-03-16T10:11:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec0b59f0 con 0x7fb2ec0b4650 2024-03-16T10:11:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb2ec154b30 con 0x7fb2ec141820 2024-03-16T10:11:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2ebfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2ec0b7650 0x7fb2ec143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2ebfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2ec0b7650 0x7fb2ec143a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55092/0 (socket says 172.21.15.17:55092) 2024-03-16T10:11:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2ebfff640 1 -- 172.21.15.17:0/1826850151 learned_addr learned my addr 172.21.15.17:0/1826850151 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:49.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f1462640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2ec141820 0x7fb2ec1e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2ec0b4650 0x7fb2ec1395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2ebfff640 1 -- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2ec141820 msgr2=0x7fb2ec1e2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2ebfff640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2ec141820 0x7fb2ec1e2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2ebfff640 1 -- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2ec0b4650 msgr2=0x7fb2ec1395a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2ebfff640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2ec0b4650 0x7fb2ec1395a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2ebfff640 1 -- 172.21.15.17:0/1826850151 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ec146d70 con 0x7fb2ec0b7650 2024-03-16T10:11:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2ec0b4650 0x7fb2ec1395a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2ebfff640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2ec0b7650 0x7fb2ec143a20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00d7d0 tx=0x7fb2dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2e97fa640 1 -- 172.21.15.17:0/1826850151 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00b6c0 con 0x7fb2ec0b7650 2024-03-16T10:11:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2e97fa640 1 -- 172.21.15.17:0/1826850151 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2dc019070 con 0x7fb2ec0b7650 2024-03-16T10:11:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 -- 172.21.15.17:0/1826850151 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2ec147670 con 0x7fb2ec0b7650 2024-03-16T10:11:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2e97fa640 1 -- 172.21.15.17:0/1826850151 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00ba80 con 0x7fb2ec0b7650 2024-03-16T10:11:49.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.428+0000 7fb2f2ec5640 1 -- 172.21.15.17:0/1826850151 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2ec146ff0 con 0x7fb2ec0b7650 2024-03-16T10:11:49.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.432+0000 7fb2e97fa640 1 -- 172.21.15.17:0/1826850151 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fb2dc00bc20 con 0x7fb2ec0b7650 2024-03-16T10:11:49.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.432+0000 7fb2e97fa640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb2d8067660 0x7fb2d8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.432+0000 7fb2e97fa640 1 -- 172.21.15.17:0/1826850151 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fb2dc085c40 con 0x7fb2ec0b7650 2024-03-16T10:11:49.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.432+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb2d8067660 0x7fb2d8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.432+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb2d8067660 0x7fb2d8069b20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0007fd0 tx=0x7fb2e0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:49.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.432+0000 7fb2f6047000 1 Processor -- start 2024-03-16T10:11:49.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.432+0000 7fb2f6047000 1 -- start start 2024-03-16T10:11:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d24694a0 0x55b6d2469880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d2469dc0 0x55b6d2494180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24946c0 0x55b6d2494aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x55b6d22b2540 con 0x55b6d24946c0 2024-03-16T10:11:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b6d2239b00 con 0x55b6d24694a0 2024-03-16T10:11:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55b6d22ae5e0 con 0x55b6d2469dc0 2024-03-16T10:11:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24946c0 0x55b6d2494aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24946c0 0x55b6d2494aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55096/0 (socket says 172.21.15.17:55096) 2024-03-16T10:11:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 -- 172.21.15.17:0/3918917937 learned_addr learned my addr 172.21.15.17:0/3918917937 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:49.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/3918917937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d24694a0 0x55b6d2469880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2ebfff640 1 --2- 172.21.15.17:0/3918917937 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d2469dc0 0x55b6d2494180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 -- 172.21.15.17:0/3918917937 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d2469dc0 msgr2=0x55b6d2494180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 --2- 172.21.15.17:0/3918917937 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d2469dc0 0x55b6d2494180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 -- 172.21.15.17:0/3918917937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d24694a0 msgr2=0x55b6d2469880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 --2- 172.21.15.17:0/3918917937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d24694a0 0x55b6d2469880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 -- 172.21.15.17:0/3918917937 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6d226f6a0 con 0x55b6d24946c0 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 --2- 172.21.15.17:0/3918917937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24946c0 0x55b6d2494aa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb2e401cdc0 tx=0x7fb2e4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd5bb5d52a9c64e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2d27fc640 1 -- 172.21.15.17:0/3918917937 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40b2040 con 0x55b6d24946c0 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2d27fc640 1 -- 172.21.15.17:0/3918917937 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e4003ba0 con 0x55b6d24946c0 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2d27fc640 1 -- 172.21.15.17:0/3918917937 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4004030 con 0x55b6d24946c0 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 -- 172.21.15.17:0/3918917937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24946c0 msgr2=0x55b6d2494aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 --2- 172.21.15.17:0/3918917937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24946c0 0x55b6d2494aa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb2e401cdc0 tx=0x7fb2e4004ac0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 -- 172.21.15.17:0/3918917937 shutdown_connections 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 --2- 172.21.15.17:0/3918917937 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24946c0 0x55b6d2494aa0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 --2- 172.21.15.17:0/3918917937 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d2469dc0 0x55b6d2494180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 --2- 172.21.15.17:0/3918917937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d24694a0 0x55b6d2469880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 -- 172.21.15.17:0/3918917937 >> 172.21.15.17:0/3918917937 conn(0x55b6d248ff30 msgr2=0x55b6d2492320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 -- 172.21.15.17:0/3918917937 shutdown_connections 2024-03-16T10:11:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 -- 172.21.15.17:0/3918917937 wait complete. 2024-03-16T10:11:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 -- start start 2024-03-16T10:11:49.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 Processor -- start 2024-03-16T10:11:49.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d24694a0 0x55b6d2462e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d2469dc0 0x55b6d2463390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24946c0 0x55b6d24638d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x55b6d22b6b70 con 0x55b6d24946c0 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b6d2270850 con 0x55b6d2469dc0 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55b6d2496020 con 0x55b6d24694a0 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f6047000 10 client.0 _ll_register_callbacks cb 0x55b6d244f220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24946c0 0x55b6d24638d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24946c0 0x55b6d24638d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55108/0 (socket says 172.21.15.17:55108) 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 -- 172.21.15.17:0/2525363387 learned_addr learned my addr 172.21.15.17:0/2525363387 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d24694a0 0x55b6d2462e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2ebfff640 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d2469dc0 0x55b6d2463390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 -- 172.21.15.17:0/2525363387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d24694a0 msgr2=0x55b6d2462e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d24694a0 0x55b6d2462e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 -- 172.21.15.17:0/2525363387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d2469dc0 msgr2=0x55b6d2463390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:49.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d2469dc0 0x55b6d2463390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f1462640 1 -- 172.21.15.17:0/2525363387 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6d226d900 con 0x55b6d24946c0 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d24694a0 0x55b6d2462e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.436+0000 7fb2ebfff640 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d2469dc0 0x55b6d2463390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f1462640 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24946c0 0x55b6d24638d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009d90 tx=0x7fb2e401b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ba06eff7537e73 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2cb7fe640 1 -- 172.21.15.17:0/2525363387 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e40c6070 con 0x55b6d24946c0 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2cb7fe640 1 -- 172.21.15.17:0/2525363387 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fb2e4003d50 con 0x55b6d24946c0 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 -- 172.21.15.17:0/2525363387 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e4003ba0 con 0x55b6d24946c0 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2cb7fe640 1 -- 172.21.15.17:0/2525363387 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7fb2e40d3050 con 0x55b6d24946c0 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 10 client.5181 fetch_fsmap learned FSMap version 15 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 -- 172.21.15.17:0/2525363387 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55b6d2333940 con 0x55b6d24946c0 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2cb7fe640 1 -- 172.21.15.17:0/2525363387 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e4003ba0 con 0x55b6d24946c0 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2cb7fe640 1 -- 172.21.15.17:0/2525363387 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 15) ==== 2284+0+0 (secure 0 0 0) 0x7fb2e40a5cb0 con 0x55b6d24946c0 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 10 client.5181 fetch_fsmap finished waiting for FSMap version 15 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 10 client.5181 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 20 client.5181 populate_metadata read hostname 'smithi017' 2024-03-16T10:11:49.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.17:6832/578943207,v1:172.21.15.17:6833/578943207] conn(0x55b6d24a7560 0x55b6d24a9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 4 client.5181 mds_command: new command op to 4953 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 -- 172.21.15.17:0/2525363387 --> [v2:172.21.15.17:6832/578943207,v1:172.21.15.17:6833/578943207] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6d224dbc0 con 0x55b6d24a7560 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.17:6832/578943207,v1:172.21.15.17:6833/578943207] conn(0x55b6d24a7560 0x55b6d24a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2cb7fe640 10 client.5181 ms_handle_connect on v2:172.21.15.17:6832/578943207 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.17:6832/578943207,v1:172.21.15.17:6833/578943207] conn(0x55b6d24a7560 0x55b6d24a9950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b64198113fc9b1b3 server_cookie=3edd7048fc7893e2 in_seq=0 out_seq=0 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2cb7fe640 1 -- 172.21.15.17:0/2525363387 <== mds.0 v2:172.21.15.17:6832/578943207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55b6d224dbc0 con 0x55b6d24a7560 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2cb7fe640 10 client.5181 handle_command_reply: tid=0 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 client.5181 shutdown 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 -- 172.21.15.17:0/2525363387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24946c0 msgr2=0x55b6d24638d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24946c0 0x55b6d24638d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009d90 tx=0x7fb2e401b120 comp rx=0 tx=0).stop 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 -- 172.21.15.17:0/2525363387 shutdown_connections 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.17:6832/578943207,v1:172.21.15.17:6833/578943207] conn(0x55b6d24a7560 0x55b6d24a9950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24946c0 0x55b6d24638d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d2469dc0 0x55b6d2463390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 --2- 172.21.15.17:0/2525363387 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d24694a0 0x55b6d2462e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 -- 172.21.15.17:0/2525363387 >> 172.21.15.17:0/2525363387 conn(0x55b6d248ff30 msgr2=0x55b6d2492320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2cb7fe640 0 client.5181 ms_handle_reset on v2:172.21.15.17:6832/578943207 2024-03-16T10:11:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 -- 172.21.15.17:0/2525363387 shutdown_connections 2024-03-16T10:11:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 1 -- 172.21.15.17:0/2525363387 wait complete. 2024-03-16T10:11:49.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.440+0000 7fb2f6047000 20 client.5181 trim_cache size 0 max 16384 2024-03-16T10:11:49.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 Processor -- start 2024-03-16T10:11:49.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 -- start start 2024-03-16T10:11:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24694a0 0x55b6d24a9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d2469dc0 0x55b6d24a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d24946c0 0x55b6d24a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec1ec380 con 0x55b6d24694a0 2024-03-16T10:11:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec1eb4f0 con 0x55b6d2469dc0 2024-03-16T10:11:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55b6d23334b0 con 0x55b6d24946c0 2024-03-16T10:11:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f0c61640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24694a0 0x55b6d24a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f0c61640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24694a0 0x55b6d24a9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55120/0 (socket says 172.21.15.17:55120) 2024-03-16T10:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f0c61640 1 -- 172.21.15.17:0/2394875651 learned_addr learned my addr 172.21.15.17:0/2394875651 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2ebfff640 1 --2- 172.21.15.17:0/2394875651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d2469dc0 0x55b6d24a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f1462640 1 --2- 172.21.15.17:0/2394875651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d24946c0 0x55b6d24a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f0c61640 1 -- 172.21.15.17:0/2394875651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d24946c0 msgr2=0x55b6d24a9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/2394875651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d24946c0 0x55b6d24a9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f0c61640 1 -- 172.21.15.17:0/2394875651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d2469dc0 msgr2=0x55b6d24a9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/2394875651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d2469dc0 0x55b6d24a9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f0c61640 1 -- 172.21.15.17:0/2394875651 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6d230a870 con 0x55b6d24694a0 2024-03-16T10:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2ebfff640 1 --2- 172.21.15.17:0/2394875651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d2469dc0 0x55b6d24a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f1462640 1 --2- 172.21.15.17:0/2394875651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d24946c0 0x55b6d24a9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/2394875651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24694a0 0x55b6d24a9570 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00071d0 tx=0x7fb2e0016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912a3ba412bc89da server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2d1ffb640 1 -- 172.21.15.17:0/2394875651 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e001d040 con 0x55b6d24694a0 2024-03-16T10:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2d1ffb640 1 -- 172.21.15.17:0/2394875651 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e0029020 con 0x55b6d24694a0 2024-03-16T10:11:49.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2d1ffb640 1 -- 172.21.15.17:0/2394875651 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e00243d0 con 0x55b6d24694a0 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 -- 172.21.15.17:0/2394875651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24694a0 msgr2=0x55b6d24a9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 --2- 172.21.15.17:0/2394875651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24694a0 0x55b6d24a9570 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00071d0 tx=0x7fb2e0016040 comp rx=0 tx=0).stop 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 -- 172.21.15.17:0/2394875651 shutdown_connections 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 --2- 172.21.15.17:0/2394875651 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d24946c0 0x55b6d24a9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 --2- 172.21.15.17:0/2394875651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d2469dc0 0x55b6d24a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 --2- 172.21.15.17:0/2394875651 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24694a0 0x55b6d24a9570 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 -- 172.21.15.17:0/2394875651 >> 172.21.15.17:0/2394875651 conn(0x55b6d248ff30 msgr2=0x55b6d2492320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 -- 172.21.15.17:0/2394875651 shutdown_connections 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 -- 172.21.15.17:0/2394875651 wait complete. 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.480+0000 7fb2f6047000 1 -- start start 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 Processor -- start 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24694a0 0x55b6d25015e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d2469dc0 0x55b6d2502260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d24946c0 0x55b6d25027a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x55b6d224a7b0 con 0x55b6d24694a0 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b6d226cf50 con 0x55b6d24946c0 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x55b6d22b1850 con 0x55b6d2469dc0 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 10 client.0 _ll_register_callbacks cb 0x7fb2e4009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f0c61640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24694a0 0x55b6d25015e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f0c61640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24694a0 0x55b6d25015e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55130/0 (socket says 172.21.15.17:55130) 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f0c61640 1 -- 172.21.15.17:0/719067139 learned_addr learned my addr 172.21.15.17:0/719067139 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f1462640 1 --2- 172.21.15.17:0/719067139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d24946c0 0x55b6d25027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2ebfff640 1 --2- 172.21.15.17:0/719067139 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d2469dc0 0x55b6d2502260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f0c61640 1 -- 172.21.15.17:0/719067139 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d2469dc0 msgr2=0x55b6d2502260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/719067139 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d2469dc0 0x55b6d2502260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f0c61640 1 -- 172.21.15.17:0/719067139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d24946c0 msgr2=0x55b6d25027a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/719067139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d24946c0 0x55b6d25027a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f0c61640 1 -- 172.21.15.17:0/719067139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6d2262070 con 0x55b6d24694a0 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f1462640 1 --2- 172.21.15.17:0/719067139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d24946c0 0x55b6d25027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f0c61640 1 --2- 172.21.15.17:0/719067139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24694a0 0x55b6d25015e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb2e002b040 tx=0x7fb2e0000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78140fcafa1098e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:49.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2cb7fe640 1 -- 172.21.15.17:0/719067139 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0032030 con 0x55b6d24694a0 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2cb7fe640 1 -- 172.21.15.17:0/719067139 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fb2e00245e0 con 0x55b6d24694a0 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 -- 172.21.15.17:0/719067139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e0029020 con 0x55b6d24694a0 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2cb7fe640 1 -- 172.21.15.17:0/719067139 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7fb2e003e050 con 0x55b6d24694a0 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 10 client.5187 fetch_fsmap learned FSMap version 15 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 -- 172.21.15.17:0/719067139 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x55b6d2246370 con 0x55b6d24694a0 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2cb7fe640 1 -- 172.21.15.17:0/719067139 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2e0029020 con 0x55b6d24694a0 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2cb7fe640 1 -- 172.21.15.17:0/719067139 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 15) ==== 2284+0+0 (secure 0 0 0) 0x7fb2e0041020 con 0x55b6d24694a0 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 10 client.5187 fetch_fsmap finished waiting for FSMap version 15 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 10 client.5187 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 20 client.5187 populate_metadata read hostname 'smithi017' 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 --2- 172.21.15.17:0/719067139 >> [v2:172.21.15.17:6832/578943207,v1:172.21.15.17:6833/578943207] conn(0x55b6d2502ce0 0x55b6d25030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 4 client.5187 mds_command: new command op to 4953 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 -- 172.21.15.17:0/719067139 --> [v2:172.21.15.17:6832/578943207,v1:172.21.15.17:6833/578943207] -- command(tid 0: {"prefix": "status"}) -- 0x55b6d22bfa80 con 0x55b6d2502ce0 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2ebfff640 1 --2- 172.21.15.17:0/719067139 >> [v2:172.21.15.17:6832/578943207,v1:172.21.15.17:6833/578943207] conn(0x55b6d2502ce0 0x55b6d25030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2ebfff640 1 --2- 172.21.15.17:0/719067139 >> [v2:172.21.15.17:6832/578943207,v1:172.21.15.17:6833/578943207] conn(0x55b6d2502ce0 0x55b6d25030c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ad08d3bf890f1b0 server_cookie=858c3068574d8a97 in_seq=0 out_seq=0 2024-03-16T10:11:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2cb7fe640 10 client.5187 ms_handle_connect on v2:172.21.15.17:6832/578943207 2024-03-16T10:11:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2cb7fe640 1 -- 172.21.15.17:0/719067139 <== mds.0 v2:172.21.15.17:6832/578943207 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55b6d22bfa80 con 0x55b6d2502ce0 2024-03-16T10:11:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2cb7fe640 10 client.5187 handle_command_reply: tid=0 2024-03-16T10:11:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 client.5187 shutdown 2024-03-16T10:11:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 -- 172.21.15.17:0/719067139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24694a0 msgr2=0x55b6d25015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 --2- 172.21.15.17:0/719067139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24694a0 0x55b6d25015e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb2e002b040 tx=0x7fb2e0000ee0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 -- 172.21.15.17:0/719067139 shutdown_connections 2024-03-16T10:11:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 --2- 172.21.15.17:0/719067139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6d24946c0 0x55b6d25027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 --2- 172.21.15.17:0/719067139 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55b6d2469dc0 0x55b6d2502260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 --2- 172.21.15.17:0/719067139 >> [v2:172.21.15.17:6832/578943207,v1:172.21.15.17:6833/578943207] conn(0x55b6d2502ce0 0x55b6d25030c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 --2- 172.21.15.17:0/719067139 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x55b6d24694a0 0x55b6d25015e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 -- 172.21.15.17:0/719067139 >> 172.21.15.17:0/719067139 conn(0x55b6d248ff30 msgr2=0x55b6d2492bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2cb7fe640 0 client.5187 ms_handle_reset on v2:172.21.15.17:6832/578943207 2024-03-16T10:11:49.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 -- 172.21.15.17:0/719067139 shutdown_connections 2024-03-16T10:11:49.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 1 -- 172.21.15.17:0/719067139 wait complete. 2024-03-16T10:11:49.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f6047000 20 client.5187 trim_cache size 0 max 16384 2024-03-16T10:11:49.496 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-16T10:11:49.496 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "b21729d9-93de-4054-b789-790eefecfc1c", 2024-03-16T10:11:49.496 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-03-16T10:11:49.496 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4953, 2024-03-16T10:11:49.496 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-03-16T10:11:49.496 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-03-16T10:11:49.496 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-03-16T10:11:49.496 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 9.4700465539999996, 2024-03-16T10:11:49.496 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 15, 2024-03-16T10:11:49.497 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 30, 2024-03-16T10:11:49.497 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 30, 2024-03-16T10:11:49.497 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 17.012255047 2024-03-16T10:11:49.497 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-16T10:11:49.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f2ec5640 1 -- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb2d8067660 msgr2=0x7fb2d8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f2ec5640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb2d8067660 0x7fb2d8069b20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0007fd0 tx=0x7fb2e0009040 comp rx=0 tx=0).stop 2024-03-16T10:11:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f2ec5640 1 -- 172.21.15.17:0/1826850151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2ec0b7650 msgr2=0x7fb2ec143a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f2ec5640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2ec0b7650 0x7fb2ec143a20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00d7d0 tx=0x7fb2dc00dc90 comp rx=0 tx=0).stop 2024-03-16T10:11:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f2ec5640 1 -- 172.21.15.17:0/1826850151 shutdown_connections 2024-03-16T10:11:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f2ec5640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb2d8067660 0x7fb2d8069b20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f2ec5640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb2ec141820 0x7fb2ec1e2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f2ec5640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2ec0b7650 0x7fb2ec143a20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f2ec5640 1 --2- 172.21.15.17:0/1826850151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2ec0b4650 0x7fb2ec1395a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f2ec5640 1 -- 172.21.15.17:0/1826850151 >> 172.21.15.17:0/1826850151 conn(0x7fb2ec0bccd0 msgr2=0x7fb2ec0baf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f2ec5640 1 -- 172.21.15.17:0/1826850151 shutdown_connections 2024-03-16T10:11:49.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.484+0000 7fb2f2ec5640 1 -- 172.21.15.17:0/1826850151 wait complete. 2024-03-16T10:11:49.515 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-16T10:11:49.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.620+0000 7fa7ee6b3640 1 Processor -- start 2024-03-16T10:11:49.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.620+0000 7fa7ee6b3640 1 -- start start 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7ee6b3640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa7e806e7b0 0x7fa7e80b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7ee6b3640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa7e813e670 0x7fa7e81440b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7ee6b3640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7e806eb90 con 0x7fa7e8070780 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7ee6b3640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e8070c30 con 0x7fa7e813e670 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7ee6b3640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fa7e81445f0 con 0x7fa7e806e7b0 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7ecc50640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa7e813e670 0x7fa7e81440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7e77fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa7e8070780 0x7fa7e813df60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42868/0 (socket says 172.21.15.17:42868) 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7ecc50640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa7e813e670 0x7fa7e81440b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55140/0 (socket says 172.21.15.17:55140) 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7e77fe640 1 -- 172.21.15.17:0/1231627612 learned_addr learned my addr 172.21.15.17:0/1231627612 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7e7fff640 1 --2- 172.21.15.17:0/1231627612 >> v2:172.21.15.174:3300/0 conn(0x7fa7e806e7b0 0x7fa7e80b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7ecc50640 1 -- 172.21.15.17:0/1231627612 >> v2:172.21.15.174:3300/0 conn(0x7fa7e806e7b0 msgr2=0x7fa7e80b5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7ecc50640 1 --2- 172.21.15.17:0/1231627612 >> v2:172.21.15.174:3300/0 conn(0x7fa7e806e7b0 0x7fa7e80b5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7ecc50640 1 -- 172.21.15.17:0/1231627612 >> v1:172.21.15.17:6789/0 conn(0x7fa7e8070780 legacy=0x7fa7e813df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7ecc50640 1 -- 172.21.15.17:0/1231627612 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e8072e60 con 0x7fa7e813e670 2024-03-16T10:11:49.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7e6ffd640 1 -- 172.21.15.17:0/1231627612 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3510134506 0 0) 0x7fa7e806eb90 con 0x7fa7e8070780 2024-03-16T10:11:49.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7ecc50640 1 --2- 172.21.15.17:0/1231627612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7e813e670 0x7fa7e81440b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8067e10 tx=0x7fa7e001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4808a1e93db314 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:49.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7e6ffd640 1 -- 172.21.15.17:0/1231627612 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00a66e0 con 0x7fa7e813e670 2024-03-16T10:11:49.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7e6ffd640 1 -- 172.21.15.17:0/1231627612 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e00a6880 con 0x7fa7e813e670 2024-03-16T10:11:49.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.624+0000 7fa7e6ffd640 1 -- 172.21.15.17:0/1231627612 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e00a6c50 con 0x7fa7e813e670 2024-03-16T10:11:49.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/1231627612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7e813e670 msgr2=0x7fa7e81440b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 --2- 172.21.15.17:0/1231627612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7e813e670 0x7fa7e81440b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8067e10 tx=0x7fa7e001c810 comp rx=0 tx=0).stop 2024-03-16T10:11:49.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/1231627612 shutdown_connections 2024-03-16T10:11:49.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 --2- 172.21.15.17:0/1231627612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7e813e670 0x7fa7e81440b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 --2- 172.21.15.17:0/1231627612 >> v2:172.21.15.174:3300/0 conn(0x7fa7e806e7b0 0x7fa7e80b5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/1231627612 >> 172.21.15.17:0/1231627612 conn(0x7fa7e80b65d0 msgr2=0x7fa7e80b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:49.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/1231627612 shutdown_connections 2024-03-16T10:11:49.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/1231627612 wait complete. 2024-03-16T10:11:49.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 Processor -- start 2024-03-16T10:11:49.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 -- start start 2024-03-16T10:11:49.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7e806e7b0 0x7fa7e8143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7e8070780 0x7fa7e813f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e813e670 0x7fa7e81e13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e814f590 con 0x7fa7e8070780 2024-03-16T10:11:49.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7e814f410 con 0x7fa7e813e670 2024-03-16T10:11:49.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa7e814f710 con 0x7fa7e806e7b0 2024-03-16T10:11:49.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7e77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7e8070780 0x7fa7e813f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7e77fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7e8070780 0x7fa7e813f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55154/0 (socket says 172.21.15.17:55154) 2024-03-16T10:11:49.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7e77fe640 1 -- 172.21.15.17:0/548638146 learned_addr learned my addr 172.21.15.17:0/548638146 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:49.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7e7fff640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7e806e7b0 0x7fa7e8143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ecc50640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e813e670 0x7fa7e81e13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7e77fe640 1 -- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7e806e7b0 msgr2=0x7fa7e8143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7e77fe640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7e806e7b0 0x7fa7e8143070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7e77fe640 1 -- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e813e670 msgr2=0x7fa7e81e13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7e77fe640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e813e670 0x7fa7e81e13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7e77fe640 1 -- 172.21.15.17:0/548638146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7e80b1e30 con 0x7fa7e8070780 2024-03-16T10:11:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ecc50640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e813e670 0x7fa7e81e13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7e77fe640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7e8070780 0x7fa7e813f200 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80011f0 tx=0x7fa7d800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7e4ff9640 1 -- 172.21.15.17:0/548638146 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8018070 con 0x7fa7e8070780 2024-03-16T10:11:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7e4ff9640 1 -- 172.21.15.17:0/548638146 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d80074e0 con 0x7fa7e8070780 2024-03-16T10:11:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7e4ff9640 1 -- 172.21.15.17:0/548638146 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8004a10 con 0x7fa7e8070780 2024-03-16T10:11:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/548638146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7e81418c0 con 0x7fa7e8070780 2024-03-16T10:11:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.628+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/548638146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7e81e0d30 con 0x7fa7e8070780 2024-03-16T10:11:49.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.632+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/548638146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7e806a130 con 0x7fa7e8070780 2024-03-16T10:11:49.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.632+0000 7fa7e4ff9640 1 -- 172.21.15.17:0/548638146 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa7d801b070 con 0x7fa7e8070780 2024-03-16T10:11:49.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.632+0000 7fa7e4ff9640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa7d4067690 0x7fa7d4069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:49.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.632+0000 7fa7e4ff9640 1 -- 172.21.15.17:0/548638146 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fa7d8084ad0 con 0x7fa7e8070780 2024-03-16T10:11:49.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.632+0000 7fa7e7fff640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa7d4067690 0x7fa7d4069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:49.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.632+0000 7fa7e7fff640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa7d4067690 0x7fa7d4069b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8135d20 tx=0x7fa7dc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:49.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.636+0000 7fa7e4ff9640 1 -- 172.21.15.17:0/548638146 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa7d8055190 con 0x7fa7e8070780 2024-03-16T10:11:49.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.832+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/548638146 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa7e813c800 con 0x7fa7e8070780 2024-03-16T10:11:49.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.832+0000 7fa7e4ff9640 1 -- 172.21.15.17:0/548638146 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7fa7d8059230 con 0x7fa7e8070780 2024-03-16T10:11:49.839 INFO:teuthology.orchestra.run.smithi017.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-16T10:11:49.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.836+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa7d4067690 msgr2=0x7fa7d4069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.836+0000 7fa7ee6b3640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa7d4067690 0x7fa7d4069b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa7e8135d20 tx=0x7fa7dc008040 comp rx=0 tx=0).stop 2024-03-16T10:11:49.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.836+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/548638146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7e8070780 msgr2=0x7fa7e813f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:49.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.836+0000 7fa7ee6b3640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7e8070780 0x7fa7e813f200 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80011f0 tx=0x7fa7d800c9b0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.836+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/548638146 shutdown_connections 2024-03-16T10:11:49.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.836+0000 7fa7ee6b3640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa7d4067690 0x7fa7d4069b50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.836+0000 7fa7ee6b3640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7e813e670 0x7fa7e81e13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.836+0000 7fa7ee6b3640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7e8070780 0x7fa7e813f200 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.836+0000 7fa7ee6b3640 1 --2- 172.21.15.17:0/548638146 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7e806e7b0 0x7fa7e8143070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:49.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.836+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/548638146 >> 172.21.15.17:0/548638146 conn(0x7fa7e80b65d0 msgr2=0x7fa7e813d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:49.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.836+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/548638146 shutdown_connections 2024-03-16T10:11:49.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:49.836+0000 7fa7ee6b3640 1 -- 172.21.15.17:0/548638146 wait complete. 2024-03-16T10:11:49.858 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T10:11:49.858 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T10:11:49.858 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi017.front.sepia.ceph.com 2024-03-16T10:11:49.858 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-16T10:11:49.858 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:49.858 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T10:11:49.858 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T10:11:49.859 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T10:11:49.859 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-16T10:11:49.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:49.859 DEBUG:teuthology.orchestra.run.smithi017:> ip addr 2024-03-16T10:11:49.864 INFO:teuthology.orchestra.run.smithi017.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 ::1/128 scope host 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:00 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f0 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:01 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f1 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:92 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f0 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.21.15.17/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-16T10:11:49.865 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft 42491sec preferred_lft 42491sec 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 fe80::ec4:7aff:fe59:d992/64 scope link 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:93 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f1 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 02:42:d8:82:b9:a0 brd ff:ff:ff:ff:ff:ff 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-16T10:11:49.866 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:11:49.867 INFO:teuthology.orchestra.run.smithi017.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-16T10:11:49.867 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:11:49.867 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-16T10:11:49.867 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-03-16T10:11:49.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:49.867 DEBUG:teuthology.orchestra.run.smithi017:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T10:11:49.921 INFO:teuthology.orchestra.run.smithi017.stdout:1 2024-03-16T10:11:49.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:49.922 DEBUG:teuthology.orchestra.run.smithi017:> route 2024-03-16T10:11:49.973 INFO:teuthology.orchestra.run.smithi017.stdout:Kernel IP routing table 2024-03-16T10:11:49.973 INFO:teuthology.orchestra.run.smithi017.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T10:11:49.973 INFO:teuthology.orchestra.run.smithi017.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-16T10:11:49.973 INFO:teuthology.orchestra.run.smithi017.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-16T10:11:49.973 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-16T10:11:49.973 INFO:teuthology.orchestra.run.smithi017.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T10:11:49.974 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-16T10:11:49.974 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-03-16T10:11:49.974 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-16T10:11:49.974 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-16T10:11:49.974 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-16T10:11:49.974 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-03-16T10:11:50.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-16T10:11:50.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:11:50.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:50.093 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-03-16T10:11:50.098 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-16T10:11:50.098 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-16T10:11:50.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:50.098 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set brx.0 up 2024-03-16T10:11:50.151 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-16T10:11:50.151 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T10:11:50.151 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:50.195 INFO:teuthology.orchestra.run.smithi017.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-16T10:11:50.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T10:11:50.196 DEBUG:teuthology.orchestra.run.smithi017:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:50.243 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-03-16T10:11:50.298 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T10:11:50.344 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-03-16T10:11:50.344 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-03-16T10:11:50.344 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/run/lock 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/fs/nfsd 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-03-16T10:11:50.345 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-03-16T10:11:50.346 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysusers.service 2024-03-16T10:11:50.346 INFO:teuthology.orchestra.run.smithi017.stdout:/run/rpc_pipefs 2024-03-16T10:11:50.346 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-03-16T10:11:50.346 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-03-16T10:11:50.346 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-0 2024-03-16T10:11:50.346 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-1 2024-03-16T10:11:50.346 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-2 2024-03-16T10:11:50.346 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-3 2024-03-16T10:11:50.346 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-03-16T10:11:50.346 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:11:50.346 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:11:50.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:50.346 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-03-16T10:11:50.391 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-16T10:11:50.391 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-16T10:11:50.441 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-03-16T10:11:50.454 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T10:11:50.501 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-03-16T10:11:50.501 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-03-16T10:11:50.501 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-03-16T10:11:50.501 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-03-16T10:11:50.501 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-03-16T10:11:50.501 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-03-16T10:11:50.501 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-03-16T10:11:50.501 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-03-16T10:11:50.501 INFO:teuthology.orchestra.run.smithi017.stdout:/run/lock 2024-03-16T10:11:50.502 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-03-16T10:11:50.502 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-03-16T10:11:50.502 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-03-16T10:11:50.502 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:11:50.502 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-03-16T10:11:50.502 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-03-16T10:11:50.502 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-03-16T10:11:50.502 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-03-16T10:11:50.502 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/fs/nfsd 2024-03-16T10:11:50.502 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-03-16T10:11:50.502 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-03-16T10:11:50.502 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysusers.service 2024-03-16T10:11:50.502 INFO:teuthology.orchestra.run.smithi017.stdout:/run/rpc_pipefs 2024-03-16T10:11:50.502 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-03-16T10:11:50.503 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-03-16T10:11:50.503 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-0 2024-03-16T10:11:50.503 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-1 2024-03-16T10:11:50.503 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-2 2024-03-16T10:11:50.503 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-3 2024-03-16T10:11:50.503 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-03-16T10:11:50.503 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:11:50.503 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:11:50.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:50.503 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-03-16T10:11:50.519 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi017.stderr:2024-03-16T10:11:50.512+0000 7f8a7636e580 -1 init, newargv = 0x55696fed9e80 newargc=15 2024-03-16T10:11:50.519 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi017.stderr:ceph-fuse[35769]: starting ceph client 2024-03-16T10:11:50.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi017.stderr:ceph-fuse[35769]: starting fuse 2024-03-16T10:11:51.624 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-03-16T10:11:51.640 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T10:11:51.689 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-03-16T10:11:51.689 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-03-16T10:11:51.689 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-03-16T10:11:51.689 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-03-16T10:11:51.689 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-03-16T10:11:51.689 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-03-16T10:11:51.689 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-03-16T10:11:51.689 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/run/lock 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/fs/nfsd 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysusers.service 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/run/rpc_pipefs 2024-03-16T10:11:51.690 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-03-16T10:11:51.691 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-03-16T10:11:51.691 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-0 2024-03-16T10:11:51.691 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-1 2024-03-16T10:11:51.691 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-2 2024-03-16T10:11:51.691 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-3 2024-03-16T10:11:51.691 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-03-16T10:11:51.691 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:11:51.691 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:11:51.691 INFO:teuthology.orchestra.run.smithi017.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:51.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:51.692 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-03-16T10:11:51.740 INFO:teuthology.orchestra.run.smithi017.stdout:45 2024-03-16T10:11:51.740 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-16T10:11:51.740 DEBUG:teuthology.orchestra.run.smithi017:> sudo stdin-killer -- python3 -c ' 2024-03-16T10:11:51.740 DEBUG:teuthology.orchestra.run.smithi017:> import glob 2024-03-16T10:11:51.740 DEBUG:teuthology.orchestra.run.smithi017:> import re 2024-03-16T10:11:51.740 DEBUG:teuthology.orchestra.run.smithi017:> import os 2024-03-16T10:11:51.740 DEBUG:teuthology.orchestra.run.smithi017:> import subprocess 2024-03-16T10:11:51.740 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-16T10:11:51.740 DEBUG:teuthology.orchestra.run.smithi017:> def _find_admin_socket(client_name): 2024-03-16T10:11:51.740 DEBUG:teuthology.orchestra.run.smithi017:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-16T10:11:51.740 DEBUG:teuthology.orchestra.run.smithi017:> files = glob.glob(asok_path) 2024-03-16T10:11:51.740 DEBUG:teuthology.orchestra.run.smithi017:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> # Given a non-glob path, it better be there 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> if "*" not in asok_path: 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> assert(len(files) == 1) 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> return files[0] 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> for f in files: 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> contents = proc_f.read() 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> if mountpoint in contents: 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> return f 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> print(_find_admin_socket("client.0")) 2024-03-16T10:11:51.741 DEBUG:teuthology.orchestra.run.smithi017:> ' 2024-03-16T10:11:51.866 INFO:teuthology.orchestra.run.smithi017.stdout:/var/run/ceph/ceph-client.0.35769.asok 2024-03-16T10:11:51.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:11:51.879 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35769.asok 2024-03-16T10:11:51.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:51.879 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35769.asok status 2024-03-16T10:11:52.036 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-16T10:11:52.036 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": { 2024-03-16T10:11:52.036 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_sha1": "3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", 2024-03-16T10:11:52.036 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-16T10:11:52.036 INFO:teuthology.orchestra.run.smithi017.stdout: "entity_id": "0", 2024-03-16T10:11:52.036 INFO:teuthology.orchestra.run.smithi017.stdout: "hostname": "smithi017", 2024-03-16T10:11:52.036 INFO:teuthology.orchestra.run.smithi017.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: "pid": "35769", 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: "root": "/" 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_count": 0, 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_pinned_count": 0, 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 5202, 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: "inst": { 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: "name": { 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "client", 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5202 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2024-03-16T10:11:52.037 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3355318968 2024-03-16T10:11:52.038 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-03-16T10:11:52.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:52.038 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2024-03-16T10:11:52.038 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-03-16T10:11:52.038 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2024-03-16T10:11:52.038 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 3355318968 2024-03-16T10:11:52.038 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-03-16T10:11:52.038 INFO:teuthology.orchestra.run.smithi017.stdout: "inst_str": "client.5202 192.168.0.1:0/3355318968", 2024-03-16T10:11:52.038 INFO:teuthology.orchestra.run.smithi017.stdout: "addr_str": "192.168.0.1:0/3355318968", 2024-03-16T10:11:52.038 INFO:teuthology.orchestra.run.smithi017.stdout: "inode_count": 1, 2024-03-16T10:11:52.038 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_epoch": 15, 2024-03-16T10:11:52.038 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch": 30, 2024-03-16T10:11:52.038 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch_barrier": 0, 2024-03-16T10:11:52.039 INFO:teuthology.orchestra.run.smithi017.stdout: "blocklisted": false, 2024-03-16T10:11:52.039 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs" 2024-03-16T10:11:52.039 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-16T10:11:52.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:52.050 DEBUG:teuthology.orchestra.run.smithi017:> stat --file-system '--printf=%T 2024-03-16T10:11:52.050 DEBUG:teuthology.orchestra.run.smithi017:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:52.056 INFO:teuthology.orchestra.run.smithi017.stdout:fuseblk 2024-03-16T10:11:52.057 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:52.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:52.057 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:52.112 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2024-03-16T10:11:52.164 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.2bPtv81HyK 2024-03-16T10:11:52.164 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:11:52.164 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/tmp/tmp.2bPtv81HyK 2024-03-16T10:11:52.212 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p dir1/dir2/dir3') 2024-03-16T10:11:52.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T10:11:52.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:11:52.376 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'chmod 755 /tmp/test-cephfs8olo1904') 2024-03-16T10:11:52.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-16T10:11:52.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T10:11:52.455 INFO:tasks.cephfs.test_cephfs_shell:Running script "/tmp/test-cephfs8olo1904" 2024-03-16T10:11:52.455 DEBUG:teuthology.orchestra.run.smithi017:> cephfs-shell -b /tmp/test-cephfs8olo1904 2024-03-16T10:11:52.817 INFO:teuthology.orchestra.run.smithi017.stderr:Unrecognized command 2024-03-16T10:11:52.847 DEBUG:teuthology.orchestra.run:got remote process result: 127 2024-03-16T10:11:52.848 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-16T10:11:52.848 DEBUG:teuthology.orchestra.run.smithi017:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-16T10:11:52.889 DEBUG:teuthology.orchestra.run.smithi174:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-16T10:11:52.940 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-16T10:11:52.940 INFO:tasks.cephfs.mount:Terminating background process 2024-03-16T10:11:52.941 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:11:52.941 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T10:11:52.947 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:53.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec628df640 1 Processor -- start 2024-03-16T10:11:53.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec628df640 1 -- start start 2024-03-16T10:11:53.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec628df640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec5c067220 0x7fec5c147770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec628df640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fec5c06e080 0x7fec5c0aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec628df640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec5c0af230 con 0x7fec5c13e130 2024-03-16T10:11:53.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec628df640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fec5c0afaf0 con 0x7fec5c06e080 2024-03-16T10:11:53.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec628df640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fec5c067600 con 0x7fec5c067220 2024-03-16T10:11:53.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec5b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fec5c06e080 0x7fec5c0aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec5b7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fec5c06e080 0x7fec5c0aff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55180/0 (socket says 172.21.15.17:55180) 2024-03-16T10:11:53.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec5b7fe640 1 -- 172.21.15.17:0/2216574900 learned_addr learned my addr 172.21.15.17:0/2216574900 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:53.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec5bfff640 1 --2- 172.21.15.17:0/2216574900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec5c067220 0x7fec5c147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec5b7fe640 1 -- 172.21.15.17:0/2216574900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec5c067220 msgr2=0x7fec5c147770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:53.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec5b7fe640 1 --2- 172.21.15.17:0/2216574900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec5c067220 0x7fec5c147770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec5b7fe640 1 -- 172.21.15.17:0/2216574900 >> v1:172.21.15.174:6789/0 conn(0x7fec5c13e130 legacy=0x7fec5c144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.108+0000 7fec5b7fe640 1 -- 172.21.15.17:0/2216574900 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5c0b5600 con 0x7fec5c06e080 2024-03-16T10:11:53.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec5b7fe640 1 --2- 172.21.15.17:0/2216574900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c06e080 0x7fec5c0aff80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fec4400bb10 tx=0x7fec4401eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b176626c6098bc16 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:53.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec5affd640 1 -- 172.21.15.17:0/2216574900 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec440b4070 con 0x7fec5c06e080 2024-03-16T10:11:53.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec5affd640 1 -- 172.21.15.17:0/2216574900 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec440a7a70 con 0x7fec5c06e080 2024-03-16T10:11:53.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec5affd640 1 -- 172.21.15.17:0/2216574900 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec440a7e40 con 0x7fec5c06e080 2024-03-16T10:11:53.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 -- 172.21.15.17:0/2216574900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c06e080 msgr2=0x7fec5c0aff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 --2- 172.21.15.17:0/2216574900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c06e080 0x7fec5c0aff80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fec4400bb10 tx=0x7fec4401eba0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 -- 172.21.15.17:0/2216574900 shutdown_connections 2024-03-16T10:11:53.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 --2- 172.21.15.17:0/2216574900 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c06e080 0x7fec5c0aff80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 --2- 172.21.15.17:0/2216574900 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec5c067220 0x7fec5c147770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 -- 172.21.15.17:0/2216574900 >> 172.21.15.17:0/2216574900 conn(0x7fec5c09b130 msgr2=0x7fec5c09b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:53.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 -- 172.21.15.17:0/2216574900 shutdown_connections 2024-03-16T10:11:53.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 -- 172.21.15.17:0/2216574900 wait complete. 2024-03-16T10:11:53.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 Processor -- start 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 -- start start 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec5c067220 0x7fec5c072580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec5c06e080 0x7fec5c143420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c13e130 0x7fec5c073120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c06cd50 con 0x7fec5c13e130 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c09c040 con 0x7fec5c06e080 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fec5c06ced0 con 0x7fec5c067220 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec60e7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c13e130 0x7fec5c073120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec60e7c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c13e130 0x7fec5c073120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55196/0 (socket says 172.21.15.17:55196) 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec60e7c640 1 -- 172.21.15.17:0/466235004 learned_addr learned my addr 172.21.15.17:0/466235004 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec5b7fe640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec5c06e080 0x7fec5c143420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec5bfff640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec5c067220 0x7fec5c072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec60e7c640 1 -- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec5c067220 msgr2=0x7fec5c072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec60e7c640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec5c067220 0x7fec5c072580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec60e7c640 1 -- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec5c06e080 msgr2=0x7fec5c143420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec60e7c640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec5c06e080 0x7fec5c143420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec60e7c640 1 -- 172.21.15.17:0/466235004 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5c09d0c0 con 0x7fec5c13e130 2024-03-16T10:11:53.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec5bfff640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec5c067220 0x7fec5c072580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:53.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec5b7fe640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec5c06e080 0x7fec5c143420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:53.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec60e7c640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c13e130 0x7fec5c073120 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00c340 tx=0x7fec4c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:53.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec58ff9640 1 -- 172.21.15.17:0/466235004 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c005220 con 0x7fec5c13e130 2024-03-16T10:11:53.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec58ff9640 1 -- 172.21.15.17:0/466235004 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec4c0053c0 con 0x7fec5c13e130 2024-03-16T10:11:53.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec58ff9640 1 -- 172.21.15.17:0/466235004 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4c007270 con 0x7fec5c13e130 2024-03-16T10:11:53.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 -- 172.21.15.17:0/466235004 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5c0720e0 con 0x7fec5c13e130 2024-03-16T10:11:53.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.112+0000 7fec628df640 1 -- 172.21.15.17:0/466235004 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5c1476f0 con 0x7fec5c13e130 2024-03-16T10:11:53.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.116+0000 7fec628df640 1 -- 172.21.15.17:0/466235004 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec5c06a130 con 0x7fec5c13e130 2024-03-16T10:11:53.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.116+0000 7fec58ff9640 1 -- 172.21.15.17:0/466235004 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fec4c019020 con 0x7fec5c13e130 2024-03-16T10:11:53.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.116+0000 7fec58ff9640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fec3c067660 0x7fec3c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.116+0000 7fec58ff9640 1 -- 172.21.15.17:0/466235004 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fec4c0851d0 con 0x7fec5c13e130 2024-03-16T10:11:53.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.116+0000 7fec5bfff640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fec3c067660 0x7fec3c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.116+0000 7fec5bfff640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fec3c067660 0x7fec3c069b20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fec5c143e40 tx=0x7fec5400b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:53.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.120+0000 7fec58ff9640 1 -- 172.21.15.17:0/466235004 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fec4c020070 con 0x7fec5c13e130 2024-03-16T10:11:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.280+0000 7fec628df640 1 -- 172.21.15.17:0/466235004 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec5c070270 con 0x7fec5c13e130 2024-03-16T10:11:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.280+0000 7fec58ff9640 1 -- 172.21.15.17:0/466235004 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16669 (secure 0 0 0) 0x7fec4c055fc0 con 0x7fec5c13e130 2024-03-16T10:11:53.287 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:53.287 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":30,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:11:41.019229+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:11:37.284106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T10:11:38.392554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:53.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.284+0000 7fec628df640 1 -- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fec3c067660 msgr2=0x7fec3c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.284+0000 7fec628df640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fec3c067660 0x7fec3c069b20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fec5c143e40 tx=0x7fec5400b040 comp rx=0 tx=0).stop 2024-03-16T10:11:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.284+0000 7fec628df640 1 -- 172.21.15.17:0/466235004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c13e130 msgr2=0x7fec5c073120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.284+0000 7fec628df640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c13e130 0x7fec5c073120 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00c340 tx=0x7fec4c00c800 comp rx=0 tx=0).stop 2024-03-16T10:11:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.284+0000 7fec628df640 1 -- 172.21.15.17:0/466235004 shutdown_connections 2024-03-16T10:11:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.284+0000 7fec628df640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fec3c067660 0x7fec3c069b20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.284+0000 7fec628df640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec5c13e130 0x7fec5c073120 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.284+0000 7fec628df640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec5c06e080 0x7fec5c143420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.284+0000 7fec628df640 1 --2- 172.21.15.17:0/466235004 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fec5c067220 0x7fec5c072580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.284+0000 7fec628df640 1 -- 172.21.15.17:0/466235004 >> 172.21.15.17:0/466235004 conn(0x7fec5c09b130 msgr2=0x7fec5c13d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.284+0000 7fec628df640 1 -- 172.21.15.17:0/466235004 shutdown_connections 2024-03-16T10:11:53.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.284+0000 7fec628df640 1 -- 172.21.15.17:0/466235004 wait complete. 2024-03-16T10:11:53.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:11:37.284106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:11:38.392554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:11:53.305 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:11:53.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f9633640 1 Processor -- start 2024-03-16T10:11:53.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f9633640 1 -- start start 2024-03-16T10:11:53.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f9633640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f24f4147fe0 0x7f24f41414d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f9633640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24f40b4e50 0x7f24f4141a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f9633640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24f40aeee0 con 0x7f24f4067220 2024-03-16T10:11:53.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f9633640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f24f40af150 con 0x7f24f4147fe0 2024-03-16T10:11:53.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f9633640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f24f4067600 con 0x7f24f40b4e50 2024-03-16T10:11:53.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f27fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f24f4147fe0 0x7f24f41414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f27fc640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f24f4147fe0 0x7f24f41414d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55204/0 (socket says 172.21.15.17:55204) 2024-03-16T10:11:53.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f27fc640 1 -- 172.21.15.17:0/2968549712 learned_addr learned my addr 172.21.15.17:0/2968549712 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:53.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f2ffd640 1 --1- 172.21.15.17:0/2968549712 >> v1:172.21.15.174:6789/0 conn(0x7f24f4067220 0x7f24f41478d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.17:43902/0 (socket says 172.21.15.17:43902) 2024-03-16T10:11:53.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f37fe640 1 --2- 172.21.15.17:0/2968549712 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24f40b4e50 0x7f24f4141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f27fc640 1 -- 172.21.15.17:0/2968549712 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24f40b4e50 msgr2=0x7f24f4141a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:53.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f27fc640 1 --2- 172.21.15.17:0/2968549712 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24f40b4e50 0x7f24f4141a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f27fc640 1 -- 172.21.15.17:0/2968549712 >> v1:172.21.15.174:6789/0 conn(0x7f24f4067220 legacy=0x7f24f41478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.436+0000 7f24f27fc640 1 -- 172.21.15.17:0/2968549712 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f40b5400 con 0x7f24f4147fe0 2024-03-16T10:11:53.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f27fc640 1 --2- 172.21.15.17:0/2968549712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f4147fe0 0x7f24f41414d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009870 tx=0x7f24e801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee71ba911f7d7fa server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:53.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f1ffb640 1 -- 172.21.15.17:0/2968549712 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e80b1070 con 0x7f24f4147fe0 2024-03-16T10:11:53.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f1ffb640 1 -- 172.21.15.17:0/2968549712 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24e80ac620 con 0x7f24f4147fe0 2024-03-16T10:11:53.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f1ffb640 1 -- 172.21.15.17:0/2968549712 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e80ac9f0 con 0x7f24f4147fe0 2024-03-16T10:11:53.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 -- 172.21.15.17:0/2968549712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f4147fe0 msgr2=0x7f24f41414d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 --2- 172.21.15.17:0/2968549712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f4147fe0 0x7f24f41414d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009870 tx=0x7f24e801c4d0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 -- 172.21.15.17:0/2968549712 shutdown_connections 2024-03-16T10:11:53.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 --2- 172.21.15.17:0/2968549712 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24f40b4e50 0x7f24f4141a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 --2- 172.21.15.17:0/2968549712 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f4147fe0 0x7f24f41414d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 -- 172.21.15.17:0/2968549712 >> 172.21.15.17:0/2968549712 conn(0x7f24f406bd70 msgr2=0x7f24f40b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:53.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 -- 172.21.15.17:0/2968549712 shutdown_connections 2024-03-16T10:11:53.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 -- 172.21.15.17:0/2968549712 wait complete. 2024-03-16T10:11:53.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 Processor -- start 2024-03-16T10:11:53.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 -- start start 2024-03-16T10:11:53.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24f4067220 0x7f24f4144460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f40b4e50 0x7f24f41422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f4147fe0 0x7f24f41429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f24f4151460 con 0x7f24f4147fe0 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24f41512e0 con 0x7f24f40b4e50 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f9633640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f24f41515e0 con 0x7f24f4067220 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f4147fe0 0x7f24f41429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f4147fe0 0x7f24f41429b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55208/0 (socket says 172.21.15.17:55208) 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f37fe640 1 -- 172.21.15.17:0/3361481509 learned_addr learned my addr 172.21.15.17:0/3361481509 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f27fc640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f40b4e50 0x7f24f41422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f2ffd640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24f4067220 0x7f24f4144460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f37fe640 1 -- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24f4067220 msgr2=0x7f24f4144460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f37fe640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24f4067220 0x7f24f4144460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f37fe640 1 -- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f40b4e50 msgr2=0x7f24f41422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f37fe640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f40b4e50 0x7f24f41422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f37fe640 1 -- 172.21.15.17:0/3361481509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f413d2a0 con 0x7f24f4147fe0 2024-03-16T10:11:53.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.440+0000 7f24f2ffd640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24f4067220 0x7f24f4144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:53.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.444+0000 7f24f37fe640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f4147fe0 0x7f24f41429b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00efc0 tx=0x7f24ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:53.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.444+0000 7f24db7fe640 1 -- 172.21.15.17:0/3361481509 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec018020 con 0x7f24f4147fe0 2024-03-16T10:11:53.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.444+0000 7f24db7fe640 1 -- 172.21.15.17:0/3361481509 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24ec014480 con 0x7f24f4147fe0 2024-03-16T10:11:53.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.444+0000 7f24f9633640 1 -- 172.21.15.17:0/3361481509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f4140ba0 con 0x7f24f4147fe0 2024-03-16T10:11:53.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.444+0000 7f24db7fe640 1 -- 172.21.15.17:0/3361481509 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ec014870 con 0x7f24f4147fe0 2024-03-16T10:11:53.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.444+0000 7f24f9633640 1 -- 172.21.15.17:0/3361481509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f41407c0 con 0x7f24f4147fe0 2024-03-16T10:11:53.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.444+0000 7f24f9633640 1 -- 172.21.15.17:0/3361481509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f406a130 con 0x7f24f4147fe0 2024-03-16T10:11:53.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.444+0000 7f24db7fe640 1 -- 172.21.15.17:0/3361481509 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f24ec01b020 con 0x7f24f4147fe0 2024-03-16T10:11:53.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.444+0000 7f24db7fe640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f24e0067660 0x7f24e0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.444+0000 7f24db7fe640 1 -- 172.21.15.17:0/3361481509 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f24ec086390 con 0x7f24f4147fe0 2024-03-16T10:11:53.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.444+0000 7f24f2ffd640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f24e0067660 0x7f24e0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.444+0000 7f24f2ffd640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f24e0067660 0x7f24e0069b20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f24e4000f70 tx=0x7f24e4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:53.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.448+0000 7f24db7fe640 1 -- 172.21.15.17:0/3361481509 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f24ec022070 con 0x7f24f4147fe0 2024-03-16T10:11:53.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.648+0000 7f24f9633640 1 -- 172.21.15.17:0/3361481509 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f24f40b90c0 con 0x7f24f4147fe0 2024-03-16T10:11:53.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.652+0000 7f24db7fe640 1 -- 172.21.15.17:0/3361481509 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) ==== 93+0+10 (secure 0 0 0) 0x7f24ec010040 con 0x7f24f4147fe0 2024-03-16T10:11:53.656 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:11:53.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.652+0000 7f24f9633640 1 -- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f24e0067660 msgr2=0x7f24e0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.652+0000 7f24f9633640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f24e0067660 0x7f24e0069b20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f24e4000f70 tx=0x7f24e4009040 comp rx=0 tx=0).stop 2024-03-16T10:11:53.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.652+0000 7f24f9633640 1 -- 172.21.15.17:0/3361481509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f4147fe0 msgr2=0x7f24f41429b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.652+0000 7f24f9633640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f4147fe0 0x7f24f41429b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f24ec00efc0 tx=0x7f24ec00c490 comp rx=0 tx=0).stop 2024-03-16T10:11:53.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.652+0000 7f24f9633640 1 -- 172.21.15.17:0/3361481509 shutdown_connections 2024-03-16T10:11:53.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.652+0000 7f24f9633640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f24e0067660 0x7f24e0069b20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.652+0000 7f24f9633640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f24f4147fe0 0x7f24f41429b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.652+0000 7f24f9633640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f40b4e50 0x7f24f41422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.652+0000 7f24f9633640 1 --2- 172.21.15.17:0/3361481509 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f24f4067220 0x7f24f4144460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.652+0000 7f24f9633640 1 -- 172.21.15.17:0/3361481509 >> 172.21.15.17:0/3361481509 conn(0x7f24f406bd70 msgr2=0x7f24f409aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:53.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.652+0000 7f24f9633640 1 -- 172.21.15.17:0/3361481509 shutdown_connections 2024-03-16T10:11:53.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.652+0000 7f24f9633640 1 -- 172.21.15.17:0/3361481509 wait complete. 2024-03-16T10:11:53.674 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:11:53.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.788+0000 7f33c2e0a640 1 Processor -- start 2024-03-16T10:11:53.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.788+0000 7f33c2e0a640 1 -- start start 2024-03-16T10:11:53.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c2e0a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f33bc0b63a0 0x7f33bc06d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c2e0a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33bc073070 0x7f33bc1410f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c2e0a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f33bc0b7b70 0x7f33bc141630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c2e0a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f33bc06ce50 con 0x7f33bc0b63a0 2024-03-16T10:11:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c2e0a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f33bc0b6780 con 0x7f33bc0b7b70 2024-03-16T10:11:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c2e0a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f33bc0b6cc0 con 0x7f33bc073070 2024-03-16T10:11:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c0ba6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f33bc0b63a0 0x7f33bc06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c0ba6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f33bc0b63a0 0x7f33bc06d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55216/0 (socket says 172.21.15.17:55216) 2024-03-16T10:11:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c0ba6640 1 -- 172.21.15.17:0/1984925333 learned_addr learned my addr 172.21.15.17:0/1984925333 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c13a7640 1 --2- 172.21.15.17:0/1984925333 >> v2:172.21.15.174:3300/0 conn(0x7f33bc0b7b70 0x7f33bc141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33bbfff640 1 --2- 172.21.15.17:0/1984925333 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33bc073070 0x7f33bc1410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c0ba6640 1 -- 172.21.15.17:0/1984925333 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33bc073070 msgr2=0x7f33bc1410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c0ba6640 1 --2- 172.21.15.17:0/1984925333 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33bc073070 0x7f33bc1410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c0ba6640 1 -- 172.21.15.17:0/1984925333 >> v2:172.21.15.174:3300/0 conn(0x7f33bc0b7b70 msgr2=0x7f33bc141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c0ba6640 1 --2- 172.21.15.17:0/1984925333 >> v2:172.21.15.174:3300/0 conn(0x7f33bc0b7b70 0x7f33bc141630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c0ba6640 1 -- 172.21.15.17:0/1984925333 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc0b6fa0 con 0x7f33bc0b63a0 2024-03-16T10:11:53.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33c0ba6640 1 --2- 172.21.15.17:0/1984925333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33bc0b63a0 0x7f33bc06d270 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f33bc072fe0 tx=0x7f33ac01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcc220e266fa5f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:53.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33bb7fe640 1 -- 172.21.15.17:0/1984925333 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0a5780 con 0x7f33bc0b63a0 2024-03-16T10:11:53.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33bb7fe640 1 -- 172.21.15.17:0/1984925333 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33ac0a5920 con 0x7f33bc0b63a0 2024-03-16T10:11:53.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.792+0000 7f33bb7fe640 1 -- 172.21.15.17:0/1984925333 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0a5cf0 con 0x7f33bc0b63a0 2024-03-16T10:11:53.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 -- 172.21.15.17:0/1984925333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33bc0b63a0 msgr2=0x7f33bc06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 --2- 172.21.15.17:0/1984925333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33bc0b63a0 0x7f33bc06d270 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f33bc072fe0 tx=0x7f33ac01c9f0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 -- 172.21.15.17:0/1984925333 shutdown_connections 2024-03-16T10:11:53.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 --2- 172.21.15.17:0/1984925333 >> v2:172.21.15.174:3300/0 conn(0x7f33bc0b7b70 0x7f33bc141630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 --2- 172.21.15.17:0/1984925333 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33bc073070 0x7f33bc1410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 --2- 172.21.15.17:0/1984925333 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33bc0b63a0 0x7f33bc06d270 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 -- 172.21.15.17:0/1984925333 >> 172.21.15.17:0/1984925333 conn(0x7f33bc06b290 msgr2=0x7f33bc06b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:53.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 -- 172.21.15.17:0/1984925333 shutdown_connections 2024-03-16T10:11:53.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 -- 172.21.15.17:0/1984925333 wait complete. 2024-03-16T10:11:53.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 Processor -- start 2024-03-16T10:11:53.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 -- start start 2024-03-16T10:11:53.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33bc073070 0x7f33bc0ba450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33bc0b63a0 0x7f33bc0b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33bc0b7b70 0x7f33bc1e3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc0b5e80 con 0x7f33bc0b63a0 2024-03-16T10:11:53.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33bc0b5d00 con 0x7f33bc073070 2024-03-16T10:11:53.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f33bc147bf0 con 0x7f33bc0b7b70 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33bbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33bc0b63a0 0x7f33bc0b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33bbfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33bc0b63a0 0x7f33bc0b26f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55232/0 (socket says 172.21.15.17:55232) 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33bbfff640 1 -- 172.21.15.17:0/469215306 learned_addr learned my addr 172.21.15.17:0/469215306 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c13a7640 1 --2- 172.21.15.17:0/469215306 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33bc0b7b70 0x7f33bc1e3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c0ba6640 1 --2- 172.21.15.17:0/469215306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33bc073070 0x7f33bc0ba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33bbfff640 1 -- 172.21.15.17:0/469215306 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33bc0b7b70 msgr2=0x7f33bc1e3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33bbfff640 1 --2- 172.21.15.17:0/469215306 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33bc0b7b70 0x7f33bc1e3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33bbfff640 1 -- 172.21.15.17:0/469215306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33bc073070 msgr2=0x7f33bc0ba450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33bbfff640 1 --2- 172.21.15.17:0/469215306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33bc073070 0x7f33bc0ba450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33bbfff640 1 -- 172.21.15.17:0/469215306 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33bc13ce30 con 0x7f33bc0b63a0 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33bbfff640 1 --2- 172.21.15.17:0/469215306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33bc0b63a0 0x7f33bc0b26f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f33b000d800 tx=0x7f33b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33b97fa640 1 -- 172.21.15.17:0/469215306 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b000b6c0 con 0x7f33bc0b63a0 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33b97fa640 1 -- 172.21.15.17:0/469215306 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b0019070 con 0x7f33bc0b63a0 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33b97fa640 1 -- 172.21.15.17:0/469215306 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b000bae0 con 0x7f33bc0b63a0 2024-03-16T10:11:53.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 -- 172.21.15.17:0/469215306 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33bc0b4ea0 con 0x7f33bc0b63a0 2024-03-16T10:11:53.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.796+0000 7f33c2e0a640 1 -- 172.21.15.17:0/469215306 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33bc13cb50 con 0x7f33bc0b63a0 2024-03-16T10:11:53.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.800+0000 7f33c2e0a640 1 -- 172.21.15.17:0/469215306 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33bc06a130 con 0x7f33bc0b63a0 2024-03-16T10:11:53.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.800+0000 7f33b97fa640 1 -- 172.21.15.17:0/469215306 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f33b000bc80 con 0x7f33bc0b63a0 2024-03-16T10:11:53.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.800+0000 7f33b97fa640 1 --2- 172.21.15.17:0/469215306 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f33a8067660 0x7f33a8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:53.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.800+0000 7f33b97fa640 1 -- 172.21.15.17:0/469215306 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f33b0086890 con 0x7f33bc0b63a0 2024-03-16T10:11:53.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.800+0000 7f33c0ba6640 1 --2- 172.21.15.17:0/469215306 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f33a8067660 0x7f33a8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:53.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.800+0000 7f33c0ba6640 1 --2- 172.21.15.17:0/469215306 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f33a8067660 0x7f33a8069b20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0ae000 tx=0x7f33ac004ac0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:53.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.804+0000 7f33b97fa640 1 -- 172.21.15.17:0/469215306 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f33b0057580 con 0x7f33bc0b63a0 2024-03-16T10:11:53.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.968+0000 7f33c2e0a640 1 -- 172.21.15.17:0/469215306 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f33bc13b740 con 0x7f33bc0b63a0 2024-03-16T10:11:53.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.972+0000 7f33b97fa640 1 -- 172.21.15.17:0/469215306 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) ==== 104+0+11 (secure 0 0 0) 0x7f33b0056e80 con 0x7f33bc0b63a0 2024-03-16T10:11:53.976 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-16T10:11:53.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.972+0000 7f33c2e0a640 1 -- 172.21.15.17:0/469215306 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f33a8067660 msgr2=0x7f33a8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.972+0000 7f33c2e0a640 1 --2- 172.21.15.17:0/469215306 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f33a8067660 0x7f33a8069b20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0ae000 tx=0x7f33ac004ac0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.972+0000 7f33c2e0a640 1 -- 172.21.15.17:0/469215306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33bc0b63a0 msgr2=0x7f33bc0b26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:53.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.972+0000 7f33c2e0a640 1 --2- 172.21.15.17:0/469215306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33bc0b63a0 0x7f33bc0b26f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f33b000d800 tx=0x7f33b000dcc0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.972+0000 7f33c2e0a640 1 -- 172.21.15.17:0/469215306 shutdown_connections 2024-03-16T10:11:53.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.972+0000 7f33c2e0a640 1 --2- 172.21.15.17:0/469215306 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f33a8067660 0x7f33a8069b20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.972+0000 7f33c2e0a640 1 --2- 172.21.15.17:0/469215306 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f33bc0b7b70 0x7f33bc1e3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.972+0000 7f33c2e0a640 1 --2- 172.21.15.17:0/469215306 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33bc0b63a0 0x7f33bc0b26f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.972+0000 7f33c2e0a640 1 --2- 172.21.15.17:0/469215306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33bc073070 0x7f33bc0ba450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:53.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.972+0000 7f33c2e0a640 1 -- 172.21.15.17:0/469215306 >> 172.21.15.17:0/469215306 conn(0x7f33bc06b290 msgr2=0x7f33bc13ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:53.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.972+0000 7f33c2e0a640 1 -- 172.21.15.17:0/469215306 shutdown_connections 2024-03-16T10:11:53.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:53.972+0000 7f33c2e0a640 1 -- 172.21.15.17:0/469215306 wait complete. 2024-03-16T10:11:53.995 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:11:54.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f82840f0640 1 Processor -- start 2024-03-16T10:11:54.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f82840f0640 1 -- start start 2024-03-16T10:11:54.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f82840f0640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f827c06b4f0 0x7f827c13f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f82840f0640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f827c145910 0x7f827c0b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f82840f0640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f827c06b8d0 con 0x7f827c0b5080 2024-03-16T10:11:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f82840f0640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f827c0b5a20 con 0x7f827c06b4f0 2024-03-16T10:11:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f82840f0640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f827c0b4910 con 0x7f827c145910 2024-03-16T10:11:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f8281e8c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f827c06b4f0 0x7f827c13f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f828168b640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f827c0b5080 0x7f827c145200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42888/0 (socket says 172.21.15.17:42888) 2024-03-16T10:11:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f828168b640 1 -- 172.21.15.17:0/398545134 learned_addr learned my addr 172.21.15.17:0/398545134 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f828268d640 1 --2- 172.21.15.17:0/398545134 >> v2:172.21.15.174:3300/0 conn(0x7f827c145910 0x7f827c0b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f8280e8a640 1 -- 172.21.15.17:0/398545134 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3506149053 0 0) 0x7f827c06b8d0 con 0x7f827c0b5080 2024-03-16T10:11:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f8280e8a640 1 -- 172.21.15.17:0/398545134 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8268003530 con 0x7f827c0b5080 2024-03-16T10:11:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f8281e8c640 1 -- 172.21.15.17:0/398545134 >> v2:172.21.15.174:3300/0 conn(0x7f827c145910 msgr2=0x7f827c0b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f8281e8c640 1 --2- 172.21.15.17:0/398545134 >> v2:172.21.15.174:3300/0 conn(0x7f827c145910 0x7f827c0b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f8281e8c640 1 -- 172.21.15.17:0/398545134 >> v1:172.21.15.17:6789/0 conn(0x7f827c0b5080 legacy=0x7f827c145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.112+0000 7f8281e8c640 1 -- 172.21.15.17:0/398545134 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c145f60 con 0x7f827c06b4f0 2024-03-16T10:11:54.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.116+0000 7f8281e8c640 1 --2- 172.21.15.17:0/398545134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827c06b4f0 0x7f827c13f1a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f827c0b7b90 tx=0x7f827801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f9f3fc0e1dd45b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:54.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.116+0000 7f8280e8a640 1 -- 172.21.15.17:0/398545134 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82780b0070 con 0x7f827c06b4f0 2024-03-16T10:11:54.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.116+0000 7f8280e8a640 1 -- 172.21.15.17:0/398545134 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82780ab600 con 0x7f827c06b4f0 2024-03-16T10:11:54.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.116+0000 7f8280e8a640 1 -- 172.21.15.17:0/398545134 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82780ab9d0 con 0x7f827c06b4f0 2024-03-16T10:11:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.116+0000 7f82840f0640 1 -- 172.21.15.17:0/398545134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827c06b4f0 msgr2=0x7f827c13f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.116+0000 7f82840f0640 1 --2- 172.21.15.17:0/398545134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827c06b4f0 0x7f827c13f1a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f827c0b7b90 tx=0x7f827801c3f0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.116+0000 7f82840f0640 1 -- 172.21.15.17:0/398545134 shutdown_connections 2024-03-16T10:11:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.116+0000 7f82840f0640 1 --2- 172.21.15.17:0/398545134 >> v2:172.21.15.174:3300/0 conn(0x7f827c145910 0x7f827c0b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.116+0000 7f82840f0640 1 --2- 172.21.15.17:0/398545134 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827c06b4f0 0x7f827c13f1a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.116+0000 7f82840f0640 1 -- 172.21.15.17:0/398545134 >> 172.21.15.17:0/398545134 conn(0x7f827c0ba0e0 msgr2=0x7f827c0ba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.116+0000 7f82840f0640 1 -- 172.21.15.17:0/398545134 shutdown_connections 2024-03-16T10:11:54.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.116+0000 7f82840f0640 1 -- 172.21.15.17:0/398545134 wait complete. 2024-03-16T10:11:54.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82840f0640 1 Processor -- start 2024-03-16T10:11:54.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82840f0640 1 -- start start 2024-03-16T10:11:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82840f0640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f827c06b4f0 0x7f827c1440c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82840f0640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827c0b5080 0x7f827c12f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82840f0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f827c145910 0x7f827c1ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82840f0640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f827c1524b0 con 0x7f827c0b5080 2024-03-16T10:11:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82840f0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f827c13f8f0 con 0x7f827c145910 2024-03-16T10:11:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82840f0640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f827c152630 con 0x7f827c06b4f0 2024-03-16T10:11:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f828168b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827c0b5080 0x7f827c12f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f828168b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827c0b5080 0x7f827c12f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55240/0 (socket says 172.21.15.17:55240) 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f828168b640 1 -- 172.21.15.17:0/3928347755 learned_addr learned my addr 172.21.15.17:0/3928347755 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f8281e8c640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f827c06b4f0 0x7f827c1440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f828268d640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f827c145910 0x7f827c1ddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f828168b640 1 -- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f827c06b4f0 msgr2=0x7f827c1440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f828168b640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f827c06b4f0 0x7f827c1440c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f828168b640 1 -- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f827c145910 msgr2=0x7f827c1ddd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f828168b640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f827c145910 0x7f827c1ddd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f828168b640 1 -- 172.21.15.17:0/3928347755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f827c141520 con 0x7f827c0b5080 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f828268d640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f827c145910 0x7f827c1ddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f8281e8c640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f827c06b4f0 0x7f827c1440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f828168b640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827c0b5080 0x7f827c12f9a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c3c0 tx=0x7f826c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82727fc640 1 -- 172.21.15.17:0/3928347755 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c017070 con 0x7f827c0b5080 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82727fc640 1 -- 172.21.15.17:0/3928347755 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f826c005cf0 con 0x7f827c0b5080 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82840f0640 1 -- 172.21.15.17:0/3928347755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f827c1dec70 con 0x7f827c0b5080 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82727fc640 1 -- 172.21.15.17:0/3928347755 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f826c0072f0 con 0x7f827c0b5080 2024-03-16T10:11:54.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82840f0640 1 -- 172.21.15.17:0/3928347755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f827c1e1a30 con 0x7f827c0b5080 2024-03-16T10:11:54.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82840f0640 1 -- 172.21.15.17:0/3928347755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f827c06a130 con 0x7f827c0b5080 2024-03-16T10:11:54.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.120+0000 7f82727fc640 1 -- 172.21.15.17:0/3928347755 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f826c01a070 con 0x7f827c0b5080 2024-03-16T10:11:54.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.124+0000 7f82727fc640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8260067660 0x7f8260069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.124+0000 7f82727fc640 1 -- 172.21.15.17:0/3928347755 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f826c084c20 con 0x7f827c0b5080 2024-03-16T10:11:54.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.124+0000 7f8281e8c640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8260067660 0x7f8260069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.124+0000 7f8281e8c640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8260067660 0x7f8260069b20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82780ad000 tx=0x7f8278008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:54.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.128+0000 7f82727fc640 1 -- 172.21.15.17:0/3928347755 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f826c055930 con 0x7f827c0b5080 2024-03-16T10:11:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.288+0000 7f82840f0640 1 -- 172.21.15.17:0/3928347755 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f827c142d20 con 0x7f827c0b5080 2024-03-16T10:11:54.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.288+0000 7f82727fc640 1 -- 172.21.15.17:0/3928347755 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) ==== 100+0+11 (secure 0 0 0) 0x7f826c055230 con 0x7f827c0b5080 2024-03-16T10:11:54.292 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-16T10:11:54.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.288+0000 7f82840f0640 1 -- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8260067660 msgr2=0x7f8260069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.288+0000 7f82840f0640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8260067660 0x7f8260069b20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f82780ad000 tx=0x7f8278008040 comp rx=0 tx=0).stop 2024-03-16T10:11:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.288+0000 7f82840f0640 1 -- 172.21.15.17:0/3928347755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827c0b5080 msgr2=0x7f827c12f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.288+0000 7f82840f0640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827c0b5080 0x7f827c12f9a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f826c00c3c0 tx=0x7f826c00c880 comp rx=0 tx=0).stop 2024-03-16T10:11:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.288+0000 7f82840f0640 1 -- 172.21.15.17:0/3928347755 shutdown_connections 2024-03-16T10:11:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.288+0000 7f82840f0640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8260067660 0x7f8260069b20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.288+0000 7f82840f0640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f827c145910 0x7f827c1ddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.288+0000 7f82840f0640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f827c0b5080 0x7f827c12f9a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.288+0000 7f82840f0640 1 --2- 172.21.15.17:0/3928347755 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f827c06b4f0 0x7f827c1440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.288+0000 7f82840f0640 1 -- 172.21.15.17:0/3928347755 >> 172.21.15.17:0/3928347755 conn(0x7f827c0ba0e0 msgr2=0x7f827c1389d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.288+0000 7f82840f0640 1 -- 172.21.15.17:0/3928347755 shutdown_connections 2024-03-16T10:11:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.288+0000 7f82840f0640 1 -- 172.21.15.17:0/3928347755 wait complete. 2024-03-16T10:11:54.310 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T10:11:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.420+0000 7f34fd3fd640 1 Processor -- start 2024-03-16T10:11:54.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.420+0000 7f34fd3fd640 1 -- start start 2024-03-16T10:11:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.420+0000 7f34fd3fd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f34f809fa30 0x7f34f81482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.420+0000 7f34fd3fd640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f806b8d0 con 0x7f34f8072ab0 2024-03-16T10:11:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.420+0000 7f34fd3fd640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34f80b4f60 con 0x7f34f806b4f0 2024-03-16T10:11:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.420+0000 7f34fd3fd640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f34f80b4d10 con 0x7f34f809fa30 2024-03-16T10:11:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.420+0000 7f34f67fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f34f8072ab0 0x7f34f8147bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42902/0 (socket says 172.21.15.17:42902) 2024-03-16T10:11:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.420+0000 7f34f67fc640 1 -- 172.21.15.17:0/2284896992 learned_addr learned my addr 172.21.15.17:0/2284896992 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.420+0000 7f34f77fe640 1 --2- 172.21.15.17:0/2284896992 >> v2:172.21.15.17:3300/0 conn(0x7f34f809fa30 0x7f34f81482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.420+0000 7f34f77fe640 1 -- 172.21.15.17:0/2284896992 >> v1:172.21.15.174:6789/0 conn(0x7f34f806b4f0 legacy=0x7f34f809f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.420+0000 7f34f77fe640 1 -- 172.21.15.17:0/2284896992 >> v1:172.21.15.17:6789/0 conn(0x7f34f8072ab0 legacy=0x7f34f8147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.420+0000 7f34f77fe640 1 -- 172.21.15.17:0/2284896992 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f80b4530 con 0x7f34f809fa30 2024-03-16T10:11:54.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.420+0000 7f34f5ffb640 1 -- 172.21.15.17:0/2284896992 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 376306086 0 0) 0x7f34f806b8d0 con 0x7f34f8072ab0 2024-03-16T10:11:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f77fe640 1 --2- 172.21.15.17:0/2284896992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34f809fa30 0x7f34f81482d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009d90 tx=0x7f34f001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a6d63d9fdd19de server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f5ffb640 1 -- 172.21.15.17:0/2284896992 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f00b2070 con 0x7f34f809fa30 2024-03-16T10:11:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f5ffb640 1 -- 172.21.15.17:0/2284896992 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f00a5820 con 0x7f34f809fa30 2024-03-16T10:11:54.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f5ffb640 1 -- 172.21.15.17:0/2284896992 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f00a5bf0 con 0x7f34f809fa30 2024-03-16T10:11:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 -- 172.21.15.17:0/2284896992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34f809fa30 msgr2=0x7f34f81482d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 --2- 172.21.15.17:0/2284896992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34f809fa30 0x7f34f81482d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009d90 tx=0x7f34f001c850 comp rx=0 tx=0).stop 2024-03-16T10:11:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 -- 172.21.15.17:0/2284896992 shutdown_connections 2024-03-16T10:11:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 --2- 172.21.15.17:0/2284896992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34f809fa30 0x7f34f81482d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 -- 172.21.15.17:0/2284896992 >> 172.21.15.17:0/2284896992 conn(0x7f34f80b9a30 msgr2=0x7f34f80b9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 -- 172.21.15.17:0/2284896992 shutdown_connections 2024-03-16T10:11:54.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 -- 172.21.15.17:0/2284896992 wait complete. 2024-03-16T10:11:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 Processor -- start 2024-03-16T10:11:54.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 -- start start 2024-03-16T10:11:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34f806b4f0 0x7f34f80b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f34f8072ab0 0x7f34f8136ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34f809fa30 0x7f34f80b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f34f8155e10 con 0x7f34f806b4f0 2024-03-16T10:11:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34f80b52d0 con 0x7f34f809fa30 2024-03-16T10:11:54.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34fd3fd640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f34f8155f90 con 0x7f34f8072ab0 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34f806b4f0 0x7f34f80b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f6ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34f806b4f0 0x7f34f80b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55264/0 (socket says 172.21.15.17:55264) 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f6ffd640 1 -- 172.21.15.17:0/1008983715 learned_addr learned my addr 172.21.15.17:0/1008983715 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f67fc640 1 --2- 172.21.15.17:0/1008983715 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f34f8072ab0 0x7f34f8136ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f77fe640 1 --2- 172.21.15.17:0/1008983715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34f809fa30 0x7f34f80b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f6ffd640 1 -- 172.21.15.17:0/1008983715 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f34f8072ab0 msgr2=0x7f34f8136ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f6ffd640 1 --2- 172.21.15.17:0/1008983715 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f34f8072ab0 0x7f34f8136ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f6ffd640 1 -- 172.21.15.17:0/1008983715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34f809fa30 msgr2=0x7f34f80b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f6ffd640 1 --2- 172.21.15.17:0/1008983715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34f809fa30 0x7f34f80b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f6ffd640 1 -- 172.21.15.17:0/1008983715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34f80b3cc0 con 0x7f34f806b4f0 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34f6ffd640 1 --2- 172.21.15.17:0/1008983715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34f806b4f0 0x7f34f80b2930 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f34e800c4d0 tx=0x7f34e800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.424+0000 7f34df7fe640 1 -- 172.21.15.17:0/1008983715 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e80052d0 con 0x7f34f806b4f0 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.428+0000 7f34df7fe640 1 -- 172.21.15.17:0/1008983715 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34e8005470 con 0x7f34f806b4f0 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.428+0000 7f34fd3fd640 1 -- 172.21.15.17:0/1008983715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34f8143a70 con 0x7f34f806b4f0 2024-03-16T10:11:54.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.428+0000 7f34df7fe640 1 -- 172.21.15.17:0/1008983715 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e8012830 con 0x7f34f806b4f0 2024-03-16T10:11:54.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.428+0000 7f34fd3fd640 1 -- 172.21.15.17:0/1008983715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34f80b2130 con 0x7f34f806b4f0 2024-03-16T10:11:54.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.428+0000 7f34fd3fd640 1 -- 172.21.15.17:0/1008983715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34f806a130 con 0x7f34f806b4f0 2024-03-16T10:11:54.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.428+0000 7f34df7fe640 1 -- 172.21.15.17:0/1008983715 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f34e80129d0 con 0x7f34f806b4f0 2024-03-16T10:11:54.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.428+0000 7f34df7fe640 1 --2- 172.21.15.17:0/1008983715 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f34e4067660 0x7f34e4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.428+0000 7f34df7fe640 1 -- 172.21.15.17:0/1008983715 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f34e8085520 con 0x7f34f806b4f0 2024-03-16T10:11:54.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.428+0000 7f34f67fc640 1 --2- 172.21.15.17:0/1008983715 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f34e4067660 0x7f34e4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.428+0000 7f34f67fc640 1 --2- 172.21.15.17:0/1008983715 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f34e4067660 0x7f34e4069b20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f34ec007fd0 tx=0x7f34ec007ba0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:54.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.432+0000 7f34df7fe640 1 -- 172.21.15.17:0/1008983715 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f34e80182b0 con 0x7f34f806b4f0 2024-03-16T10:11:54.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.600+0000 7f34fd3fd640 1 -- 172.21.15.17:0/1008983715 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f34f80731f0 con 0x7f34f806b4f0 2024-03-16T10:11:54.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.600+0000 7f34df7fe640 1 -- 172.21.15.17:0/1008983715 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) ==== 80+0+0 (secure 0 0 0) 0x7f34e80561b0 con 0x7f34f806b4f0 2024-03-16T10:11:54.606 INFO:teuthology.orchestra.run.smithi017.stderr:listed 0 entries 2024-03-16T10:11:54.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.604+0000 7f34fd3fd640 1 -- 172.21.15.17:0/1008983715 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f34e4067660 msgr2=0x7f34e4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.604+0000 7f34fd3fd640 1 --2- 172.21.15.17:0/1008983715 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f34e4067660 0x7f34e4069b20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f34ec007fd0 tx=0x7f34ec007ba0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.604+0000 7f34fd3fd640 1 -- 172.21.15.17:0/1008983715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34f806b4f0 msgr2=0x7f34f80b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.604+0000 7f34fd3fd640 1 --2- 172.21.15.17:0/1008983715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34f806b4f0 0x7f34f80b2930 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f34e800c4d0 tx=0x7f34e800c990 comp rx=0 tx=0).stop 2024-03-16T10:11:54.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.604+0000 7f34fd3fd640 1 -- 172.21.15.17:0/1008983715 shutdown_connections 2024-03-16T10:11:54.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.604+0000 7f34fd3fd640 1 --2- 172.21.15.17:0/1008983715 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f34e4067660 0x7f34e4069b20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.604+0000 7f34fd3fd640 1 --2- 172.21.15.17:0/1008983715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34f809fa30 0x7f34f80b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.604+0000 7f34fd3fd640 1 --2- 172.21.15.17:0/1008983715 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f34f8072ab0 0x7f34f8136ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.604+0000 7f34fd3fd640 1 --2- 172.21.15.17:0/1008983715 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f34f806b4f0 0x7f34f80b2930 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.604+0000 7f34fd3fd640 1 -- 172.21.15.17:0/1008983715 >> 172.21.15.17:0/1008983715 conn(0x7f34f80b9a30 msgr2=0x7f34f80b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:54.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.604+0000 7f34fd3fd640 1 -- 172.21.15.17:0/1008983715 shutdown_connections 2024-03-16T10:11:54.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.604+0000 7f34fd3fd640 1 -- 172.21.15.17:0/1008983715 wait complete. 2024-03-16T10:11:54.630 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi017.front.sepia.ceph.com... 2024-03-16T10:11:54.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:54.630 DEBUG:teuthology.orchestra.run.smithi017:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-16T10:11:54.640 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi017.stderr:ceph-fuse[35769]: fuse finished with error 0 and tester_r 0 2024-03-16T10:11:54.642 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi017.front.sepia.ceph.com 2024-03-16T10:11:54.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:11:54.642 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-16T10:11:54.711 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:11:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.868+0000 7f3aec346640 1 Processor -- start 2024-03-16T10:11:54.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.868+0000 7f3aec346640 1 -- start start 2024-03-16T10:11:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ae4072c60 0x7f3ae41487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ae40b48f0 0x7f3ae413f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f3ae413f7c0 0x7f3ae40b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae4073040 con 0x7f3ae4072c60 2024-03-16T10:11:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae4148e50 con 0x7f3ae413f7c0 2024-03-16T10:11:54.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae4149130 con 0x7f3ae40b48f0 2024-03-16T10:11:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aea0e2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ae4072c60 0x7f3ae41487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aea0e2640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f3ae4072c60 0x7f3ae41487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55272/0 (socket says 172.21.15.17:55272) 2024-03-16T10:11:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aea0e2640 1 -- 172.21.15.17:0/893619482 learned_addr learned my addr 172.21.15.17:0/893619482 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aea0e2640 1 -- 172.21.15.17:0/893619482 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ae40b48f0 msgr2=0x7f3ae413f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aea0e2640 1 --2- 172.21.15.17:0/893619482 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ae40b48f0 0x7f3ae413f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aea0e2640 1 -- 172.21.15.17:0/893619482 >> v2:172.21.15.174:3300/0 conn(0x7f3ae413f7c0 msgr2=0x7f3ae40b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aea8e3640 1 --2- 172.21.15.17:0/893619482 >> v2:172.21.15.174:3300/0 conn(0x7f3ae413f7c0 0x7f3ae40b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aea0e2640 1 --2- 172.21.15.17:0/893619482 >> v2:172.21.15.174:3300/0 conn(0x7f3ae413f7c0 0x7f3ae40b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aea0e2640 1 -- 172.21.15.17:0/893619482 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae40af350 con 0x7f3ae4072c60 2024-03-16T10:11:54.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aea8e3640 1 --2- 172.21.15.17:0/893619482 >> v2:172.21.15.174:3300/0 conn(0x7f3ae413f7c0 0x7f3ae40b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aea0e2640 1 --2- 172.21.15.17:0/893619482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae4072c60 0x7f3ae41487c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40b64d0 tx=0x7f3ae001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f6f4f5cebdbccd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3ae90e0640 1 -- 172.21.15.17:0/893619482 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae00b0070 con 0x7f3ae4072c60 2024-03-16T10:11:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3ae90e0640 1 -- 172.21.15.17:0/893619482 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae00a4e50 con 0x7f3ae4072c60 2024-03-16T10:11:54.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3ae90e0640 1 -- 172.21.15.17:0/893619482 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae00ab820 con 0x7f3ae4072c60 2024-03-16T10:11:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 -- 172.21.15.17:0/893619482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae4072c60 msgr2=0x7f3ae41487c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 --2- 172.21.15.17:0/893619482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae4072c60 0x7f3ae41487c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40b64d0 tx=0x7f3ae001c400 comp rx=0 tx=0).stop 2024-03-16T10:11:54.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 -- 172.21.15.17:0/893619482 shutdown_connections 2024-03-16T10:11:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 --2- 172.21.15.17:0/893619482 >> v2:172.21.15.174:3300/0 conn(0x7f3ae413f7c0 0x7f3ae40b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 --2- 172.21.15.17:0/893619482 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ae40b48f0 0x7f3ae413f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 --2- 172.21.15.17:0/893619482 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae4072c60 0x7f3ae41487c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 -- 172.21.15.17:0/893619482 >> 172.21.15.17:0/893619482 conn(0x7f3ae40b9a30 msgr2=0x7f3ae40b9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 -- 172.21.15.17:0/893619482 shutdown_connections 2024-03-16T10:11:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.872+0000 7f3aec346640 1 -- 172.21.15.17:0/893619482 wait complete. 2024-03-16T10:11:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aec346640 1 Processor -- start 2024-03-16T10:11:54.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aec346640 1 -- start start 2024-03-16T10:11:54.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aec346640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae4072c60 0x7f3ae409f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aec346640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ae40b48f0 0x7f3ae4136a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aec346640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ae413f7c0 0x7f3ae41332b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aec346640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae406be30 con 0x7f3ae4072c60 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aec346640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae406bcb0 con 0x7f3ae413f7c0 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aec346640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae406bfb0 con 0x7f3ae40b48f0 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aea0e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae4072c60 0x7f3ae409f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aea0e2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae4072c60 0x7f3ae409f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55286/0 (socket says 172.21.15.17:55286) 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aea0e2640 1 -- 172.21.15.17:0/3947809575 learned_addr learned my addr 172.21.15.17:0/3947809575 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aea8e3640 1 --2- 172.21.15.17:0/3947809575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ae413f7c0 0x7f3ae41332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3ae98e1640 1 --2- 172.21.15.17:0/3947809575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ae40b48f0 0x7f3ae4136a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aea0e2640 1 -- 172.21.15.17:0/3947809575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ae40b48f0 msgr2=0x7f3ae4136a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aea0e2640 1 --2- 172.21.15.17:0/3947809575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ae40b48f0 0x7f3ae4136a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aea0e2640 1 -- 172.21.15.17:0/3947809575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ae413f7c0 msgr2=0x7f3ae41332b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aea0e2640 1 --2- 172.21.15.17:0/3947809575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ae413f7c0 0x7f3ae41332b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aea0e2640 1 -- 172.21.15.17:0/3947809575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae406db80 con 0x7f3ae4072c60 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aea0e2640 1 --2- 172.21.15.17:0/3947809575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae4072c60 0x7f3ae409f0b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00ae040 tx=0x7f3ae00191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3adaffd640 1 -- 172.21.15.17:0/3947809575 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae00b8020 con 0x7f3ae4072c60 2024-03-16T10:11:54.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3adaffd640 1 -- 172.21.15.17:0/3947809575 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ae00aa720 con 0x7f3ae4072c60 2024-03-16T10:11:54.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aec346640 1 -- 172.21.15.17:0/3947809575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae4136f90 con 0x7f3ae4072c60 2024-03-16T10:11:54.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3adaffd640 1 -- 172.21.15.17:0/3947809575 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae00b0040 con 0x7f3ae4072c60 2024-03-16T10:11:54.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aec346640 1 -- 172.21.15.17:0/3947809575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae4135de0 con 0x7f3ae4072c60 2024-03-16T10:11:54.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.876+0000 7f3aec346640 1 -- 172.21.15.17:0/3947809575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae406a130 con 0x7f3ae4072c60 2024-03-16T10:11:54.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.880+0000 7f3adaffd640 1 -- 172.21.15.17:0/3947809575 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f3ae00bb020 con 0x7f3ae4072c60 2024-03-16T10:11:54.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.880+0000 7f3adaffd640 1 --2- 172.21.15.17:0/3947809575 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3ac8067660 0x7f3ac8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:54.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.880+0000 7f3adaffd640 1 -- 172.21.15.17:0/3947809575 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f3ae011fd20 con 0x7f3ae4072c60 2024-03-16T10:11:54.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.880+0000 7f3ae98e1640 1 --2- 172.21.15.17:0/3947809575 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3ac8067660 0x7f3ac8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:54.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.880+0000 7f3ae98e1640 1 --2- 172.21.15.17:0/3947809575 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3ac8067660 0x7f3ac8069b20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4006fd0 tx=0x7f3ad4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:54.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:54.884+0000 7f3adaffd640 1 -- 172.21.15.17:0/3947809575 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3ae00c2070 con 0x7f3ae4072c60 2024-03-16T10:11:55.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.072+0000 7f3aec346640 1 -- 172.21.15.17:0/3947809575 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ae4132240 con 0x7f3ae4072c60 2024-03-16T10:11:55.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.076+0000 7f3adaffd640 1 -- 172.21.15.17:0/3947809575 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+3531 (secure 0 0 0) 0x7f3ae0016110 con 0x7f3ae4072c60 2024-03-16T10:11:55.080 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:55.080 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4738,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1674220578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1674220578},{"type":"v1","addr":"172.21.15.174:6835","nonce":1674220578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:11:39.392755+0000","modified":"2024-03-16T10:11:41.419952+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6833/578943207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":578943207},{"type":"v1","addr":"172.21.15.17:6833","nonce":578943207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4953,"qdb_cluster":[4953]},"id":2}]} 2024-03-16T10:11:55.080 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 15 2024-03-16T10:11:55.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.076+0000 7f3aec346640 1 -- 172.21.15.17:0/3947809575 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3ac8067660 msgr2=0x7f3ac8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.076+0000 7f3aec346640 1 --2- 172.21.15.17:0/3947809575 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3ac8067660 0x7f3ac8069b20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4006fd0 tx=0x7f3ad4008040 comp rx=0 tx=0).stop 2024-03-16T10:11:55.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.076+0000 7f3aec346640 1 -- 172.21.15.17:0/3947809575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae4072c60 msgr2=0x7f3ae409f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.076+0000 7f3aec346640 1 --2- 172.21.15.17:0/3947809575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae4072c60 0x7f3ae409f0b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3ae00ae040 tx=0x7f3ae00191a0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.076+0000 7f3aec346640 1 -- 172.21.15.17:0/3947809575 shutdown_connections 2024-03-16T10:11:55.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.076+0000 7f3aec346640 1 --2- 172.21.15.17:0/3947809575 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f3ac8067660 0x7f3ac8069b20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.076+0000 7f3aec346640 1 --2- 172.21.15.17:0/3947809575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ae413f7c0 0x7f3ae41332b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.076+0000 7f3aec346640 1 --2- 172.21.15.17:0/3947809575 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f3ae40b48f0 0x7f3ae4136a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.076+0000 7f3aec346640 1 --2- 172.21.15.17:0/3947809575 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3ae4072c60 0x7f3ae409f0b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.076+0000 7f3aec346640 1 -- 172.21.15.17:0/3947809575 >> 172.21.15.17:0/3947809575 conn(0x7f3ae40b9a30 msgr2=0x7f3ae413d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:55.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.076+0000 7f3aec346640 1 -- 172.21.15.17:0/3947809575 shutdown_connections 2024-03-16T10:11:55.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.076+0000 7f3aec346640 1 -- 172.21.15.17:0/3947809575 wait complete. 2024-03-16T10:11:55.097 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:55.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac95ee640 1 Processor -- start 2024-03-16T10:11:55.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac95ee640 1 -- start start 2024-03-16T10:11:55.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac95ee640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1ac4067220 0x7f1ac409bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac95ee640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f1ac4141940 0x7f1ac4151c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac95ee640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ac4067600 con 0x7f1ac41414c0 2024-03-16T10:11:55.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac95ee640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f1ac40b4ae0 con 0x7f1ac4067220 2024-03-16T10:11:55.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac95ee640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f1ac40b40f0 con 0x7f1ac4141940 2024-03-16T10:11:55.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1ac4067220 0x7f1ac409bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac27fc640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f1ac41414c0 0x7f1ac41474f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42918/0 (socket says 172.21.15.17:42918) 2024-03-16T10:11:55.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac2ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f1ac4067220 0x7f1ac409bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55290/0 (socket says 172.21.15.17:55290) 2024-03-16T10:11:55.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac27fc640 1 -- 172.21.15.17:0/2589774889 learned_addr learned my addr 172.21.15.17:0/2589774889 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:55.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac37fe640 1 --2- 172.21.15.17:0/2589774889 >> v2:172.21.15.174:3300/0 conn(0x7f1ac4141940 0x7f1ac4151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac2ffd640 1 -- 172.21.15.17:0/2589774889 >> v2:172.21.15.174:3300/0 conn(0x7f1ac4141940 msgr2=0x7f1ac4151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:55.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac2ffd640 1 --2- 172.21.15.17:0/2589774889 >> v2:172.21.15.174:3300/0 conn(0x7f1ac4141940 0x7f1ac4151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac2ffd640 1 -- 172.21.15.17:0/2589774889 >> v1:172.21.15.17:6789/0 conn(0x7f1ac41414c0 legacy=0x7f1ac41474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac2ffd640 1 -- 172.21.15.17:0/2589774889 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac40b4770 con 0x7f1ac4067220 2024-03-16T10:11:55.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac1ffb640 1 -- 172.21.15.17:0/2589774889 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2384948325 0 0) 0x7f1ac4067600 con 0x7f1ac41414c0 2024-03-16T10:11:55.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac2ffd640 1 --2- 172.21.15.17:0/2589774889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac4067220 0x7f1ac409bb50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40b56f0 tx=0x7f1ab401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5388b9039c33a1cb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:55.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.212+0000 7f1ac1ffb640 1 -- 172.21.15.17:0/2589774889 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40b0070 con 0x7f1ac4067220 2024-03-16T10:11:55.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac1ffb640 1 -- 172.21.15.17:0/2589774889 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab40ab600 con 0x7f1ac4067220 2024-03-16T10:11:55.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac1ffb640 1 -- 172.21.15.17:0/2589774889 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40ab9d0 con 0x7f1ac4067220 2024-03-16T10:11:55.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 -- 172.21.15.17:0/2589774889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac4067220 msgr2=0x7f1ac409bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 --2- 172.21.15.17:0/2589774889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac4067220 0x7f1ac409bb50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1ac40b56f0 tx=0x7f1ab401c3f0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 -- 172.21.15.17:0/2589774889 shutdown_connections 2024-03-16T10:11:55.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 --2- 172.21.15.17:0/2589774889 >> v2:172.21.15.174:3300/0 conn(0x7f1ac4141940 0x7f1ac4151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 --2- 172.21.15.17:0/2589774889 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac4067220 0x7f1ac409bb50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 -- 172.21.15.17:0/2589774889 >> 172.21.15.17:0/2589774889 conn(0x7f1ac40b5e10 msgr2=0x7f1ac40b61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 -- 172.21.15.17:0/2589774889 shutdown_connections 2024-03-16T10:11:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 -- 172.21.15.17:0/2589774889 wait complete. 2024-03-16T10:11:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 Processor -- start 2024-03-16T10:11:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 -- start start 2024-03-16T10:11:55.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ac4067220 0x7f1ac409e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac41414c0 0x7f1ac4142800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ac4141940 0x7f1ac41e3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4154d80 con 0x7f1ac41414c0 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4154c00 con 0x7f1ac4067220 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac4154f00 con 0x7f1ac4141940 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac41414c0 0x7f1ac4142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac27fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac41414c0 0x7f1ac4142800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55298/0 (socket says 172.21.15.17:55298) 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac27fc640 1 -- 172.21.15.17:0/3993027218 learned_addr learned my addr 172.21.15.17:0/3993027218 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac2ffd640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ac4067220 0x7f1ac409e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac37fe640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ac4141940 0x7f1ac41e3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac27fc640 1 -- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ac4141940 msgr2=0x7f1ac41e3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac27fc640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ac4141940 0x7f1ac41e3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac27fc640 1 -- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ac4067220 msgr2=0x7f1ac409e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac27fc640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ac4067220 0x7f1ac409e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac27fc640 1 -- 172.21.15.17:0/3993027218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac4142fc0 con 0x7f1ac41414c0 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac2ffd640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ac4067220 0x7f1ac409e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac27fc640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac41414c0 0x7f1ac4142800 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0011f0 tx=0x7f1aac00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1abb7fe640 1 -- 172.21.15.17:0/3993027218 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac018070 con 0x7f1ac41414c0 2024-03-16T10:11:55.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1abb7fe640 1 -- 172.21.15.17:0/3993027218 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aac0074e0 con 0x7f1ac41414c0 2024-03-16T10:11:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 -- 172.21.15.17:0/3993027218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac409f0b0 con 0x7f1ac41414c0 2024-03-16T10:11:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1abb7fe640 1 -- 172.21.15.17:0/3993027218 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac004a10 con 0x7f1ac41414c0 2024-03-16T10:11:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.216+0000 7f1ac95ee640 1 -- 172.21.15.17:0/3993027218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac40b3500 con 0x7f1ac41414c0 2024-03-16T10:11:55.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.220+0000 7f1ac95ee640 1 -- 172.21.15.17:0/3993027218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac406a130 con 0x7f1ac41414c0 2024-03-16T10:11:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.220+0000 7f1abb7fe640 1 -- 172.21.15.17:0/3993027218 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f1aac01b070 con 0x7f1ac41414c0 2024-03-16T10:11:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.220+0000 7f1abb7fe640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1aa4067660 0x7f1aa4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.220+0000 7f1abb7fe640 1 -- 172.21.15.17:0/3993027218 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f1aac084c80 con 0x7f1ac41414c0 2024-03-16T10:11:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.220+0000 7f1ac2ffd640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1aa4067660 0x7f1aa4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.220+0000 7f1ac2ffd640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1aa4067660 0x7f1aa4069b20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40ad000 tx=0x7f1ab4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:55.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.224+0000 7f1abb7fe640 1 -- 172.21.15.17:0/3993027218 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1aac055870 con 0x7f1ac41414c0 2024-03-16T10:11:55.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.384+0000 7f1ac95ee640 1 -- 172.21.15.17:0/3993027218 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ac40b36d0 con 0x7f1ac41414c0 2024-03-16T10:11:55.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.388+0000 7f1abb7fe640 1 -- 172.21.15.17:0/3993027218 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16669 (secure 0 0 0) 0x7f1aac055170 con 0x7f1ac41414c0 2024-03-16T10:11:55.392 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:55.393 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":30,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:11:41.019229+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:11:37.284106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T10:11:38.392554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:55.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.388+0000 7f1ac95ee640 1 -- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1aa4067660 msgr2=0x7f1aa4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.388+0000 7f1ac95ee640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1aa4067660 0x7f1aa4069b20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40ad000 tx=0x7f1ab4008040 comp rx=0 tx=0).stop 2024-03-16T10:11:55.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.388+0000 7f1ac95ee640 1 -- 172.21.15.17:0/3993027218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac41414c0 msgr2=0x7f1ac4142800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.388+0000 7f1ac95ee640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac41414c0 0x7f1ac4142800 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0011f0 tx=0x7f1aac00c9b0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.388+0000 7f1ac95ee640 1 -- 172.21.15.17:0/3993027218 shutdown_connections 2024-03-16T10:11:55.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.388+0000 7f1ac95ee640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f1aa4067660 0x7f1aa4069b20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.388+0000 7f1ac95ee640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f1ac4141940 0x7f1ac41e3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.388+0000 7f1ac95ee640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac41414c0 0x7f1ac4142800 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.388+0000 7f1ac95ee640 1 --2- 172.21.15.17:0/3993027218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ac4067220 0x7f1ac409e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.388+0000 7f1ac95ee640 1 -- 172.21.15.17:0/3993027218 >> 172.21.15.17:0/3993027218 conn(0x7f1ac40b5e10 msgr2=0x7f1ac4146d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:55.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.388+0000 7f1ac95ee640 1 -- 172.21.15.17:0/3993027218 shutdown_connections 2024-03-16T10:11:55.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.388+0000 7f1ac95ee640 1 -- 172.21.15.17:0/3993027218 wait complete. 2024-03-16T10:11:55.411 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T10:10:46.093125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T10:11:37.284106+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T10:11:38.392554+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T10:11:55.412 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T10:11:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3c1a37640 1 Processor -- start 2024-03-16T10:11:55.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3c1a37640 1 -- start start 2024-03-16T10:11:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3c1a37640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff3bc067220 0x7ff3bc1488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3c1a37640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3bc09bd00 0x7ff3bc09d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3c1a37640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3bc067600 con 0x7ff3bc148df0 2024-03-16T10:11:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3c1a37640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7ff3bc09e820 con 0x7ff3bc067220 2024-03-16T10:11:55.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3c1a37640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff3bc0b42b0 con 0x7ff3bc09bd00 2024-03-16T10:11:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3ba7fc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3bc09bd00 0x7ff3bc09d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3ba7fc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3bc09bd00 0x7ff3bc09d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.17:47208/0 (socket says 172.21.15.17:47208) 2024-03-16T10:11:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3baffd640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7ff3bc067220 0x7ff3bc1488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3ba7fc640 1 -- 172.21.15.17:0/1232942951 learned_addr learned my addr 172.21.15.17:0/1232942951 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3baffd640 1 -- 172.21.15.17:0/1232942951 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3bc09bd00 msgr2=0x7ff3bc09d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3baffd640 1 --2- 172.21.15.17:0/1232942951 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3bc09bd00 0x7ff3bc09d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3baffd640 1 -- 172.21.15.17:0/1232942951 >> v1:172.21.15.174:6789/0 conn(0x7ff3bc148df0 legacy=0x7ff3bc09daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3baffd640 1 -- 172.21.15.17:0/1232942951 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3bc0b55d0 con 0x7ff3bc067220 2024-03-16T10:11:55.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.532+0000 7ff3ba7fc640 1 --2- 172.21.15.17:0/1232942951 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3bc09bd00 0x7ff3bc09d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:11:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3baffd640 1 --2- 172.21.15.17:0/1232942951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc067220 0x7ff3bc1488b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40097c0 tx=0x7ff3a401ce40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=222bfc45e2cf22f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3b9ffb640 1 -- 172.21.15.17:0/1232942951 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40ab600 con 0x7ff3bc067220 2024-03-16T10:11:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3b9ffb640 1 -- 172.21.15.17:0/1232942951 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3a40b0070 con 0x7ff3bc067220 2024-03-16T10:11:55.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3b9ffb640 1 -- 172.21.15.17:0/1232942951 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3a40ab9c0 con 0x7ff3bc067220 2024-03-16T10:11:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 -- 172.21.15.17:0/1232942951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc067220 msgr2=0x7ff3bc1488b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 --2- 172.21.15.17:0/1232942951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc067220 0x7ff3bc1488b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40097c0 tx=0x7ff3a401ce40 comp rx=0 tx=0).stop 2024-03-16T10:11:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 -- 172.21.15.17:0/1232942951 shutdown_connections 2024-03-16T10:11:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 --2- 172.21.15.17:0/1232942951 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3bc09bd00 0x7ff3bc09d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 --2- 172.21.15.17:0/1232942951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc067220 0x7ff3bc1488b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 -- 172.21.15.17:0/1232942951 >> 172.21.15.17:0/1232942951 conn(0x7ff3bc06bd70 msgr2=0x7ff3bc0b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 -- 172.21.15.17:0/1232942951 shutdown_connections 2024-03-16T10:11:55.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 -- 172.21.15.17:0/1232942951 wait complete. 2024-03-16T10:11:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 Processor -- start 2024-03-16T10:11:55.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 -- start start 2024-03-16T10:11:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc067220 0x7ff3bc0b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3bc1e3510 0x7ff3bc1434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3bc143a30 0x7ff3bc0b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc09af00 con 0x7ff3bc1e3510 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc09ad80 con 0x7ff3bc067220 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.536+0000 7ff3c1a37640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7ff3bc152400 con 0x7ff3bc143a30 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3ba7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3bc1e3510 0x7ff3bc1434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3ba7fc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3bc1e3510 0x7ff3bc1434f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55306/0 (socket says 172.21.15.17:55306) 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3ba7fc640 1 -- 172.21.15.17:0/916527818 learned_addr learned my addr 172.21.15.17:0/916527818 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3baffd640 1 --2- 172.21.15.17:0/916527818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc067220 0x7ff3bc0b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3bb7fe640 1 --2- 172.21.15.17:0/916527818 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3bc143a30 0x7ff3bc0b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3ba7fc640 1 -- 172.21.15.17:0/916527818 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3bc143a30 msgr2=0x7ff3bc0b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3ba7fc640 1 --2- 172.21.15.17:0/916527818 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3bc143a30 0x7ff3bc0b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3ba7fc640 1 -- 172.21.15.17:0/916527818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc067220 msgr2=0x7ff3bc0b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3ba7fc640 1 --2- 172.21.15.17:0/916527818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc067220 0x7ff3bc0b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3ba7fc640 1 -- 172.21.15.17:0/916527818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3bc071910 con 0x7ff3bc1e3510 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3ba7fc640 1 --2- 172.21.15.17:0/916527818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3bc1e3510 0x7ff3bc1434f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00ea30 tx=0x7ff3ac00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3b3fff640 1 -- 172.21.15.17:0/916527818 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac017070 con 0x7ff3bc1e3510 2024-03-16T10:11:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3b3fff640 1 -- 172.21.15.17:0/916527818 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3ac00cac0 con 0x7ff3bc1e3510 2024-03-16T10:11:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3c1a37640 1 -- 172.21.15.17:0/916527818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3bc070250 con 0x7ff3bc1e3510 2024-03-16T10:11:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3b3fff640 1 -- 172.21.15.17:0/916527818 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac00ce70 con 0x7ff3bc1e3510 2024-03-16T10:11:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3c1a37640 1 -- 172.21.15.17:0/916527818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3bc071630 con 0x7ff3bc1e3510 2024-03-16T10:11:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3c1a37640 1 -- 172.21.15.17:0/916527818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3bc06a130 con 0x7ff3bc1e3510 2024-03-16T10:11:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3b3fff640 1 -- 172.21.15.17:0/916527818 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7ff3ac01a070 con 0x7ff3bc1e3510 2024-03-16T10:11:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3b3fff640 1 --2- 172.21.15.17:0/916527818 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff39c067660 0x7ff39c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3b3fff640 1 -- 172.21.15.17:0/916527818 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7ff3ac085b60 con 0x7ff3bc1e3510 2024-03-16T10:11:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3baffd640 1 --2- 172.21.15.17:0/916527818 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff39c067660 0x7ff39c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.540+0000 7ff3baffd640 1 --2- 172.21.15.17:0/916527818 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff39c067660 0x7ff39c069b20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40af000 tx=0x7ff3a4004740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:55.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.552+0000 7ff3b3fff640 1 -- 172.21.15.17:0/916527818 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff3ac0567f0 con 0x7ff3bc1e3510 2024-03-16T10:11:55.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.704+0000 7ff3c1a37640 1 -- 172.21.15.17:0/916527818 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff3bc0b3740 con 0x7ff3bc1e3510 2024-03-16T10:11:55.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.708+0000 7ff3b3fff640 1 -- 172.21.15.17:0/916527818 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) ==== 93+0+10 (secure 0 0 0) 0x7ff3ac0560f0 con 0x7ff3bc1e3510 2024-03-16T10:11:55.712 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-03-16T10:11:55.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.708+0000 7ff3c1a37640 1 -- 172.21.15.17:0/916527818 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff39c067660 msgr2=0x7ff39c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.708+0000 7ff3c1a37640 1 --2- 172.21.15.17:0/916527818 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff39c067660 0x7ff39c069b20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff3a40af000 tx=0x7ff3a4004740 comp rx=0 tx=0).stop 2024-03-16T10:11:55.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.708+0000 7ff3c1a37640 1 -- 172.21.15.17:0/916527818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3bc1e3510 msgr2=0x7ff3bc1434f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.708+0000 7ff3c1a37640 1 --2- 172.21.15.17:0/916527818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3bc1e3510 0x7ff3bc1434f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac00ea30 tx=0x7ff3ac00eef0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.708+0000 7ff3c1a37640 1 -- 172.21.15.17:0/916527818 shutdown_connections 2024-03-16T10:11:55.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.708+0000 7ff3c1a37640 1 --2- 172.21.15.17:0/916527818 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7ff39c067660 0x7ff39c069b20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.708+0000 7ff3c1a37640 1 --2- 172.21.15.17:0/916527818 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7ff3bc143a30 0x7ff3bc0b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.708+0000 7ff3c1a37640 1 --2- 172.21.15.17:0/916527818 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3bc1e3510 0x7ff3bc1434f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.708+0000 7ff3c1a37640 1 --2- 172.21.15.17:0/916527818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc067220 0x7ff3bc0b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.708+0000 7ff3c1a37640 1 -- 172.21.15.17:0/916527818 >> 172.21.15.17:0/916527818 conn(0x7ff3bc06bd70 msgr2=0x7ff3bc14e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:55.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.708+0000 7ff3c1a37640 1 -- 172.21.15.17:0/916527818 shutdown_connections 2024-03-16T10:11:55.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.708+0000 7ff3c1a37640 1 -- 172.21.15.17:0/916527818 wait complete. 2024-03-16T10:11:55.729 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T10:11:55.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad9192640 1 Processor -- start 2024-03-16T10:11:55.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad9192640 1 -- start start 2024-03-16T10:11:55.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad9192640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fdad406e3c0 0x7fdad40af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad9192640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdad40af8d0 0x7fdad4147d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad9192640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad406e7a0 con 0x7fdad41482a0 2024-03-16T10:11:55.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad9192640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fdad409f260 con 0x7fdad40af8d0 2024-03-16T10:11:55.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad9192640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fdad40aec20 con 0x7fdad406e3c0 2024-03-16T10:11:55.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdad40af8d0 0x7fdad4147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad259c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fdad40af8d0 0x7fdad4147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55316/0 (socket says 172.21.15.17:55316) 2024-03-16T10:11:55.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad259c640 1 -- 172.21.15.17:0/678330657 learned_addr learned my addr 172.21.15.17:0/678330657 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:55.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad2d9d640 1 --2- 172.21.15.17:0/678330657 >> v2:172.21.15.174:3300/0 conn(0x7fdad406e3c0 0x7fdad40af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad259c640 1 -- 172.21.15.17:0/678330657 >> v2:172.21.15.174:3300/0 conn(0x7fdad406e3c0 msgr2=0x7fdad40af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:55.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad259c640 1 --2- 172.21.15.17:0/678330657 >> v2:172.21.15.174:3300/0 conn(0x7fdad406e3c0 0x7fdad40af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad259c640 1 -- 172.21.15.17:0/678330657 >> v1:172.21.15.174:6789/0 conn(0x7fdad41482a0 legacy=0x7fdad40b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:55.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.860+0000 7fdad259c640 1 -- 172.21.15.17:0/678330657 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad406dd10 con 0x7fdad40af8d0 2024-03-16T10:11:55.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.864+0000 7fdad259c640 1 --2- 172.21.15.17:0/678330657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad40af8d0 0x7fdad4147d60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdad40b99e0 tx=0x7fdabc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec9f88980940bbb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:55.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.864+0000 7fdad1d9b640 1 -- 172.21.15.17:0/678330657 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0b0070 con 0x7fdad40af8d0 2024-03-16T10:11:55.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.864+0000 7fdad1d9b640 1 -- 172.21.15.17:0/678330657 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdabc0ab600 con 0x7fdad40af8d0 2024-03-16T10:11:55.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.864+0000 7fdad1d9b640 1 -- 172.21.15.17:0/678330657 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0ab9d0 con 0x7fdad40af8d0 2024-03-16T10:11:55.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.864+0000 7fdad9192640 1 -- 172.21.15.17:0/678330657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad40af8d0 msgr2=0x7fdad4147d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.864+0000 7fdad9192640 1 --2- 172.21.15.17:0/678330657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad40af8d0 0x7fdad4147d60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdad40b99e0 tx=0x7fdabc01c4e0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.864+0000 7fdad9192640 1 -- 172.21.15.17:0/678330657 shutdown_connections 2024-03-16T10:11:55.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.864+0000 7fdad9192640 1 --2- 172.21.15.17:0/678330657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad40af8d0 0x7fdad4147d60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.864+0000 7fdad9192640 1 --2- 172.21.15.17:0/678330657 >> v2:172.21.15.174:3300/0 conn(0x7fdad406e3c0 0x7fdad40af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.864+0000 7fdad9192640 1 -- 172.21.15.17:0/678330657 >> 172.21.15.17:0/678330657 conn(0x7fdad40b65d0 msgr2=0x7fdad40b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:55.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.864+0000 7fdad9192640 1 -- 172.21.15.17:0/678330657 shutdown_connections 2024-03-16T10:11:55.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.864+0000 7fdad9192640 1 -- 172.21.15.17:0/678330657 wait complete. 2024-03-16T10:11:55.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad9192640 1 Processor -- start 2024-03-16T10:11:55.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad9192640 1 -- start start 2024-03-16T10:11:55.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad9192640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdad406e3c0 0x7fdad4145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad9192640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdad40af8d0 0x7fdad4142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad9192640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad41482a0 0x7fdad4146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad9192640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdad406f070 con 0x7fdad41482a0 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad9192640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdad406b8e0 con 0x7fdad40af8d0 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad9192640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fdad406d4d0 con 0x7fdad406e3c0 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad41482a0 0x7fdad4146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad359e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad41482a0 0x7fdad4146510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55326/0 (socket says 172.21.15.17:55326) 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad359e640 1 -- 172.21.15.17:0/747359354 learned_addr learned my addr 172.21.15.17:0/747359354 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad259c640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdad40af8d0 0x7fdad4142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad2d9d640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdad406e3c0 0x7fdad4145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad359e640 1 -- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdad406e3c0 msgr2=0x7fdad4145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad359e640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdad406e3c0 0x7fdad4145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad359e640 1 -- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdad40af8d0 msgr2=0x7fdad4142ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad359e640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdad40af8d0 0x7fdad4142ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad359e640 1 -- 172.21.15.17:0/747359354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad4149ab0 con 0x7fdad41482a0 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad259c640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdad40af8d0 0x7fdad4142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad2d9d640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdad406e3c0 0x7fdad4145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:55.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad359e640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad41482a0 0x7fdad4146510 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdac400b410 tx=0x7fdac400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:55.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdacb7fe640 1 -- 172.21.15.17:0/747359354 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4015020 con 0x7fdad41482a0 2024-03-16T10:11:55.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdacb7fe640 1 -- 172.21.15.17:0/747359354 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac4011870 con 0x7fdad41482a0 2024-03-16T10:11:55.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdacb7fe640 1 -- 172.21.15.17:0/747359354 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4011c60 con 0x7fdad41482a0 2024-03-16T10:11:55.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad9192640 1 -- 172.21.15.17:0/747359354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad41444a0 con 0x7fdad41482a0 2024-03-16T10:11:55.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad9192640 1 -- 172.21.15.17:0/747359354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad4144a60 con 0x7fdad41482a0 2024-03-16T10:11:55.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdad9192640 1 -- 172.21.15.17:0/747359354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad406a130 con 0x7fdad41482a0 2024-03-16T10:11:55.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdacb7fe640 1 -- 172.21.15.17:0/747359354 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fdac4018020 con 0x7fdad41482a0 2024-03-16T10:11:55.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdacb7fe640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fdab4067660 0x7fdab4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:55.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.868+0000 7fdacb7fe640 1 -- 172.21.15.17:0/747359354 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fdac4085400 con 0x7fdad41482a0 2024-03-16T10:11:55.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.872+0000 7fdad2d9d640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fdab4067660 0x7fdab4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:55.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.872+0000 7fdad2d9d640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fdab4067660 0x7fdab4069b20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdacc007330 tx=0x7fdacc00b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:55.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:55.876+0000 7fdacb7fe640 1 -- 172.21.15.17:0/747359354 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdac401f030 con 0x7fdad41482a0 2024-03-16T10:11:56.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.084+0000 7fdad9192640 1 -- 172.21.15.17:0/747359354 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdad4145fa0 con 0x7fdad41482a0 2024-03-16T10:11:56.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.088+0000 7fdacb7fe640 1 -- 172.21.15.17:0/747359354 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) ==== 104+0+11 (secure 0 0 0) 0x7fdac4018310 con 0x7fdad41482a0 2024-03-16T10:11:56.092 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-03-16T10:11:56.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.088+0000 7fdad9192640 1 -- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fdab4067660 msgr2=0x7fdab4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.088+0000 7fdad9192640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fdab4067660 0x7fdab4069b20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fdacc007330 tx=0x7fdacc00b040 comp rx=0 tx=0).stop 2024-03-16T10:11:56.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.088+0000 7fdad9192640 1 -- 172.21.15.17:0/747359354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad41482a0 msgr2=0x7fdad4146510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.088+0000 7fdad9192640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad41482a0 0x7fdad4146510 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdac400b410 tx=0x7fdac400b8d0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.088+0000 7fdad9192640 1 -- 172.21.15.17:0/747359354 shutdown_connections 2024-03-16T10:11:56.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.088+0000 7fdad9192640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fdab4067660 0x7fdab4069b20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.088+0000 7fdad9192640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad41482a0 0x7fdad4146510 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.088+0000 7fdad9192640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdad40af8d0 0x7fdad4142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.088+0000 7fdad9192640 1 --2- 172.21.15.17:0/747359354 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fdad406e3c0 0x7fdad4145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.088+0000 7fdad9192640 1 -- 172.21.15.17:0/747359354 >> 172.21.15.17:0/747359354 conn(0x7fdad40b65d0 msgr2=0x7fdad40b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:56.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.088+0000 7fdad9192640 1 -- 172.21.15.17:0/747359354 shutdown_connections 2024-03-16T10:11:56.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.088+0000 7fdad9192640 1 -- 172.21.15.17:0/747359354 wait complete. 2024-03-16T10:11:56.111 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T10:11:56.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0cbb70640 1 Processor -- start 2024-03-16T10:11:56.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0cbb70640 1 -- start start 2024-03-16T10:11:56.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0cbb70640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb0c4067220 0x7fb0c41478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0cbb70640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c40b4e50 0x7fb0c4147e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0cbb70640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb0c41414d0 0x7fb0c41418e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0cbb70640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb0c40af150 con 0x7fb0c4067220 2024-03-16T10:11:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0cbb70640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fb0c40aeee0 con 0x7fb0c41414d0 2024-03-16T10:11:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0cbb70640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c4067600 con 0x7fb0c40b4e50 2024-03-16T10:11:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0c990c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb0c4067220 0x7fb0c41478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0c990c640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb0c4067220 0x7fb0c41478d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55342/0 (socket says 172.21.15.17:55342) 2024-03-16T10:11:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0c990c640 1 -- 172.21.15.17:0/4285211654 learned_addr learned my addr 172.21.15.17:0/4285211654 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0c910b640 1 --2- 172.21.15.17:0/4285211654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c40b4e50 0x7fb0c4147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0ca10d640 1 --2- 172.21.15.17:0/4285211654 >> v2:172.21.15.174:3300/0 conn(0x7fb0c41414d0 0x7fb0c41418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0c990c640 1 -- 172.21.15.17:0/4285211654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c40b4e50 msgr2=0x7fb0c4147e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0c990c640 1 --2- 172.21.15.17:0/4285211654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c40b4e50 0x7fb0c4147e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0c990c640 1 -- 172.21.15.17:0/4285211654 >> v2:172.21.15.174:3300/0 conn(0x7fb0c41414d0 msgr2=0x7fb0c41418e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0c990c640 1 --2- 172.21.15.17:0/4285211654 >> v2:172.21.15.174:3300/0 conn(0x7fb0c41414d0 0x7fb0c41418e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.244+0000 7fb0c990c640 1 -- 172.21.15.17:0/4285211654 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c40b5400 con 0x7fb0c4067220 2024-03-16T10:11:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0c990c640 1 --2- 172.21.15.17:0/4285211654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c4067220 0x7fb0c41478d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40ae460 tx=0x7fb0bc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5a2a6a1fc835d4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0c890a640 1 -- 172.21.15.17:0/4285211654 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc0b2070 con 0x7fb0c4067220 2024-03-16T10:11:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0c890a640 1 -- 172.21.15.17:0/4285211654 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0bc0a9070 con 0x7fb0c4067220 2024-03-16T10:11:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0c890a640 1 -- 172.21.15.17:0/4285211654 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc0ad880 con 0x7fb0c4067220 2024-03-16T10:11:56.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 -- 172.21.15.17:0/4285211654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c4067220 msgr2=0x7fb0c41478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 --2- 172.21.15.17:0/4285211654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c4067220 0x7fb0c41478d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40ae460 tx=0x7fb0bc01c490 comp rx=0 tx=0).stop 2024-03-16T10:11:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 -- 172.21.15.17:0/4285211654 shutdown_connections 2024-03-16T10:11:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 --2- 172.21.15.17:0/4285211654 >> v2:172.21.15.174:3300/0 conn(0x7fb0c41414d0 0x7fb0c41418e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 --2- 172.21.15.17:0/4285211654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c40b4e50 0x7fb0c4147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 --2- 172.21.15.17:0/4285211654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c4067220 0x7fb0c41478d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 -- 172.21.15.17:0/4285211654 >> 172.21.15.17:0/4285211654 conn(0x7fb0c406bd70 msgr2=0x7fb0c40b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 -- 172.21.15.17:0/4285211654 shutdown_connections 2024-03-16T10:11:56.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 -- 172.21.15.17:0/4285211654 wait complete. 2024-03-16T10:11:56.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 Processor -- start 2024-03-16T10:11:56.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 -- start start 2024-03-16T10:11:56.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c4067220 0x7fb0c409c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c40b4e50 0x7fb0c409d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c41414d0 0x7fb0c41dfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c409e890 con 0x7fb0c41414d0 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c409e710 con 0x7fb0c4067220 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb0c40afdf0 con 0x7fb0c40b4e50 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0ca10d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c41414d0 0x7fb0c41dfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0ca10d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c41414d0 0x7fb0c41dfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55358/0 (socket says 172.21.15.17:55358) 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0ca10d640 1 -- 172.21.15.17:0/965806442 learned_addr learned my addr 172.21.15.17:0/965806442 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0c910b640 1 --2- 172.21.15.17:0/965806442 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c40b4e50 0x7fb0c409d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0c990c640 1 --2- 172.21.15.17:0/965806442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c4067220 0x7fb0c409c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0ca10d640 1 -- 172.21.15.17:0/965806442 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c40b4e50 msgr2=0x7fb0c409d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0ca10d640 1 --2- 172.21.15.17:0/965806442 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c40b4e50 0x7fb0c409d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0ca10d640 1 -- 172.21.15.17:0/965806442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c4067220 msgr2=0x7fb0c409c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0ca10d640 1 --2- 172.21.15.17:0/965806442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c4067220 0x7fb0c409c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0ca10d640 1 -- 172.21.15.17:0/965806442 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c414e680 con 0x7fb0c41414d0 2024-03-16T10:11:56.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0ca10d640 1 --2- 172.21.15.17:0/965806442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c41414d0 0x7fb0c41dfe80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0007e60 tx=0x7fb0c000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:56.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0ba7fc640 1 -- 172.21.15.17:0/965806442 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0014420 con 0x7fb0c41414d0 2024-03-16T10:11:56.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0ba7fc640 1 -- 172.21.15.17:0/965806442 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0c0019070 con 0x7fb0c41414d0 2024-03-16T10:11:56.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0ba7fc640 1 -- 172.21.15.17:0/965806442 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0014820 con 0x7fb0c41414d0 2024-03-16T10:11:56.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 -- 172.21.15.17:0/965806442 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c409d6d0 con 0x7fb0c41414d0 2024-03-16T10:11:56.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.248+0000 7fb0cbb70640 1 -- 172.21.15.17:0/965806442 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c40b8c40 con 0x7fb0c41414d0 2024-03-16T10:11:56.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.252+0000 7fb0cbb70640 1 -- 172.21.15.17:0/965806442 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c406a130 con 0x7fb0c41414d0 2024-03-16T10:11:56.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.252+0000 7fb0ba7fc640 1 -- 172.21.15.17:0/965806442 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fb0c00149c0 con 0x7fb0c41414d0 2024-03-16T10:11:56.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.252+0000 7fb0ba7fc640 1 --2- 172.21.15.17:0/965806442 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 0x7fb0a8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.252+0000 7fb0ba7fc640 1 -- 172.21.15.17:0/965806442 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fb0c0086070 con 0x7fb0c41414d0 2024-03-16T10:11:56.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.252+0000 7fb0c990c640 1 --2- 172.21.15.17:0/965806442 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 0x7fb0a8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.252+0000 7fb0c990c640 1 --2- 172.21.15.17:0/965806442 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 0x7fb0a8069b20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0af000 tx=0x7fb0bc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:56.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.256+0000 7fb0ba7fc640 1 -- 172.21.15.17:0/965806442 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb0c409d6d0 con 0x7fb0c41414d0 2024-03-16T10:11:56.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.420+0000 7fb0cbb70640 1 -- 172.21.15.17:0/965806442 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb0c409b340 con 0x7fb0c41414d0 2024-03-16T10:11:56.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.420+0000 7fb0ba7fc640 1 -- 172.21.15.17:0/965806442 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) ==== 100+0+11 (secure 0 0 0) 0x7fb0c0056ce0 con 0x7fb0c41414d0 2024-03-16T10:11:56.424 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-03-16T10:11:56.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.420+0000 7fb0cbb70640 1 -- 172.21.15.17:0/965806442 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 msgr2=0x7fb0a8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.420+0000 7fb0cbb70640 1 --2- 172.21.15.17:0/965806442 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 0x7fb0a8069b20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc0af000 tx=0x7fb0bc008040 comp rx=0 tx=0).stop 2024-03-16T10:11:56.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.420+0000 7fb0cbb70640 1 -- 172.21.15.17:0/965806442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c41414d0 msgr2=0x7fb0c41dfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.420+0000 7fb0cbb70640 1 --2- 172.21.15.17:0/965806442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c41414d0 0x7fb0c41dfe80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0007e60 tx=0x7fb0c000e4b0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.420+0000 7fb0cbb70640 1 -- 172.21.15.17:0/965806442 shutdown_connections 2024-03-16T10:11:56.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.420+0000 7fb0cbb70640 1 --2- 172.21.15.17:0/965806442 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 0x7fb0a8069b20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.420+0000 7fb0cbb70640 1 --2- 172.21.15.17:0/965806442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0c41414d0 0x7fb0c41dfe80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.420+0000 7fb0cbb70640 1 --2- 172.21.15.17:0/965806442 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0c40b4e50 0x7fb0c409d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.420+0000 7fb0cbb70640 1 --2- 172.21.15.17:0/965806442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0c4067220 0x7fb0c409c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.420+0000 7fb0cbb70640 1 -- 172.21.15.17:0/965806442 >> 172.21.15.17:0/965806442 conn(0x7fb0c406bd70 msgr2=0x7fb0c409aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:56.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.420+0000 7fb0cbb70640 1 -- 172.21.15.17:0/965806442 shutdown_connections 2024-03-16T10:11:56.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.420+0000 7fb0cbb70640 1 -- 172.21.15.17:0/965806442 wait complete. 2024-03-16T10:11:56.444 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:11:56.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c56140640 1 Processor -- start 2024-03-16T10:11:56.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c56140640 1 -- start start 2024-03-16T10:11:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c56140640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c50148a30 0x7f2c50148e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c56140640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2c5006e660 0x7f2c50149350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c56140640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c5006f5a0 con 0x7f2c500b72e0 2024-03-16T10:11:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c56140640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f2c5006fdc0 con 0x7f2c5006e660 2024-03-16T10:11:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c56140640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2c500a0e20 con 0x7f2c50148a30 2024-03-16T10:11:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4effd640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2c500b72e0 0x7f2c50142360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42932/0 (socket says 172.21.15.17:42932) 2024-03-16T10:11:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4effd640 1 -- 172.21.15.17:0/4164722322 learned_addr learned my addr 172.21.15.17:0/4164722322 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4f7fe640 1 --2- 172.21.15.17:0/4164722322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c50148a30 0x7f2c50148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4ffff640 1 --2- 172.21.15.17:0/4164722322 >> v2:172.21.15.174:3300/0 conn(0x7f2c5006e660 0x7f2c50149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 -- 172.21.15.17:0/4164722322 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2225707231 0 0) 0x7f2c5006f5a0 con 0x7f2c500b72e0 2024-03-16T10:11:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 -- 172.21.15.17:0/4164722322 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c38003530 con 0x7f2c500b72e0 2024-03-16T10:11:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 -- 172.21.15.17:0/4164722322 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2560559353 0 0) 0x7f2c38003530 con 0x7f2c500b72e0 2024-03-16T10:11:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 -- 172.21.15.17:0/4164722322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c50148a30 msgr2=0x7f2c50148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 --2- 172.21.15.17:0/4164722322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c50148a30 0x7f2c50148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 -- 172.21.15.17:0/4164722322 >> v2:172.21.15.174:3300/0 conn(0x7f2c5006e660 msgr2=0x7f2c50149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 --2- 172.21.15.17:0/4164722322 >> v2:172.21.15.174:3300/0 conn(0x7f2c5006e660 0x7f2c50149350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 -- 172.21.15.17:0/4164722322 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c50142a70 con 0x7f2c500b72e0 2024-03-16T10:11:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 -- 172.21.15.17:0/4164722322 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f2c44001680 con 0x7f2c500b72e0 2024-03-16T10:11:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 -- 172.21.15.17:0/4164722322 >> v1:172.21.15.17:6789/0 conn(0x7f2c500b72e0 legacy=0x7f2c50142360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 --2- 172.21.15.17:0/4164722322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c50148a30 0x7f2c380039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 --2- 172.21.15.17:0/4164722322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c38003f40 0x7f2c38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 --2- 172.21.15.17:0/4164722322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c38004d50 0x7f2c380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 -- 172.21.15.17:0/4164722322 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2c5006fdc0 con 0x7f2c38004d50 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 -- 172.21.15.17:0/4164722322 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c500a0e20 con 0x7f2c38003f40 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4e7fc640 1 -- 172.21.15.17:0/4164722322 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3800bab0 con 0x7f2c50148a30 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4ffff640 1 --2- 172.21.15.17:0/4164722322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c38004d50 0x7f2c380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4effd640 1 --2- 172.21.15.17:0/4164722322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c38003f40 0x7f2c38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4f7fe640 1 --2- 172.21.15.17:0/4164722322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c50148a30 0x7f2c380039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4ffff640 1 -- 172.21.15.17:0/4164722322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c50148a30 msgr2=0x7f2c380039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4ffff640 1 --2- 172.21.15.17:0/4164722322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c50148a30 0x7f2c380039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4ffff640 1 -- 172.21.15.17:0/4164722322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c38003f40 msgr2=0x7f2c38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4ffff640 1 --2- 172.21.15.17:0/4164722322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c38003f40 0x7f2c38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4ffff640 1 -- 172.21.15.17:0/4164722322 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c38003530 con 0x7f2c38004d50 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4effd640 1 --2- 172.21.15.17:0/4164722322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c38003f40 0x7f2c38004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.560+0000 7f2c4f7fe640 1 --2- 172.21.15.17:0/4164722322 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c50148a30 0x7f2c380039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:56.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4ffff640 1 --2- 172.21.15.17:0/4164722322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c38004d50 0x7f2c380051d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2c5006ea60 tx=0x7f2c4801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6582a9ee34bb992a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:56.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4e7fc640 1 -- 172.21.15.17:0/4164722322 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c480b2070 con 0x7f2c38004d50 2024-03-16T10:11:56.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4e7fc640 1 -- 172.21.15.17:0/4164722322 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c480a5b20 con 0x7f2c38004d50 2024-03-16T10:11:56.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4e7fc640 1 -- 172.21.15.17:0/4164722322 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48009c10 con 0x7f2c38004d50 2024-03-16T10:11:56.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c56140640 1 -- 172.21.15.17:0/4164722322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c38004d50 msgr2=0x7f2c380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c56140640 1 --2- 172.21.15.17:0/4164722322 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c38004d50 0x7f2c380051d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2c5006ea60 tx=0x7f2c4801cc20 comp rx=0 tx=0).stop 2024-03-16T10:11:56.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4ffff640 1 -- 172.21.15.17:0/4164722322 reap_dead start 2024-03-16T10:11:56.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c56140640 1 -- 172.21.15.17:0/4164722322 shutdown_connections 2024-03-16T10:11:56.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c56140640 1 -- 172.21.15.17:0/4164722322 >> 172.21.15.17:0/4164722322 conn(0x7f2c500718a0 msgr2=0x7f2c50071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:56.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c56140640 1 -- 172.21.15.17:0/4164722322 shutdown_connections 2024-03-16T10:11:56.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c56140640 1 -- 172.21.15.17:0/4164722322 wait complete. 2024-03-16T10:11:56.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c56140640 1 Processor -- start 2024-03-16T10:11:56.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c56140640 1 -- start start 2024-03-16T10:11:56.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c56140640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c50147ac0 0x7f2c5014de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c56140640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5014d1d0 0x7f2c5014e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c56140640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c50134f40 0x7f2c501353c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c56140640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2c50150520 con 0x7f2c50147ac0 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c56140640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c500a0fa0 con 0x7f2c50134f40 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c56140640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2c501506a0 con 0x7f2c5014d1d0 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c50147ac0 0x7f2c5014de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c50147ac0 0x7f2c5014de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55378/0 (socket says 172.21.15.17:55378) 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4f7fe640 1 -- 172.21.15.17:0/3556229440 learned_addr learned my addr 172.21.15.17:0/3556229440 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4effd640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5014d1d0 0x7f2c5014e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4ffff640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c50134f40 0x7f2c501353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4f7fe640 1 -- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5014d1d0 msgr2=0x7f2c5014e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4f7fe640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5014d1d0 0x7f2c5014e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4f7fe640 1 -- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c50134f40 msgr2=0x7f2c501353c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4f7fe640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c50134f40 0x7f2c501353c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4f7fe640 1 -- 172.21.15.17:0/3556229440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c50135900 con 0x7f2c50147ac0 2024-03-16T10:11:56.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.564+0000 7f2c4ffff640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c50134f40 0x7f2c501353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:56.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.568+0000 7f2c4f7fe640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c50147ac0 0x7f2c5014de90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2c40010900 tx=0x7f2c40010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:56.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.568+0000 7f2c37fff640 1 -- 172.21.15.17:0/3556229440 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4000f040 con 0x7f2c50147ac0 2024-03-16T10:11:56.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.568+0000 7f2c37fff640 1 -- 172.21.15.17:0/3556229440 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c40004220 con 0x7f2c50147ac0 2024-03-16T10:11:56.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.568+0000 7f2c37fff640 1 -- 172.21.15.17:0/3556229440 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4000e9c0 con 0x7f2c50147ac0 2024-03-16T10:11:56.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.568+0000 7f2c56140640 1 -- 172.21.15.17:0/3556229440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c50147ea0 con 0x7f2c50147ac0 2024-03-16T10:11:56.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.568+0000 7f2c56140640 1 -- 172.21.15.17:0/3556229440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c5014da00 con 0x7f2c50147ac0 2024-03-16T10:11:56.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.568+0000 7f2c56140640 1 -- 172.21.15.17:0/3556229440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c5006a130 con 0x7f2c50147ac0 2024-03-16T10:11:56.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.568+0000 7f2c37fff640 1 -- 172.21.15.17:0/3556229440 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f2c4000eb60 con 0x7f2c50147ac0 2024-03-16T10:11:56.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.568+0000 7f2c37fff640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2c3c067660 0x7f2c3c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.568+0000 7f2c37fff640 1 -- 172.21.15.17:0/3556229440 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f2c400857e0 con 0x7f2c50147ac0 2024-03-16T10:11:56.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.568+0000 7f2c4effd640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2c3c067660 0x7f2c3c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.572+0000 7f2c4effd640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2c3c067660 0x7f2c3c069b20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2c500b9f60 tx=0x7f2c44007040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:56.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.572+0000 7f2c37fff640 1 -- 172.21.15.17:0/3556229440 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2c40018030 con 0x7f2c50147ac0 2024-03-16T10:11:56.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.768+0000 7f2c56140640 1 -- 172.21.15.17:0/3556229440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c500ba310 con 0x7f2c50147ac0 2024-03-16T10:11:56.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.772+0000 7f2c37fff640 1 -- 172.21.15.17:0/3556229440 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+3531 (secure 0 0 0) 0x7f2c40055ea0 con 0x7f2c50147ac0 2024-03-16T10:11:56.776 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:56.776 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4738,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1674220578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1674220578},{"type":"v1","addr":"172.21.15.174:6835","nonce":1674220578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:11:39.392755+0000","modified":"2024-03-16T10:11:41.419952+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6833/578943207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":578943207},{"type":"v1","addr":"172.21.15.17:6833","nonce":578943207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4953,"qdb_cluster":[4953]},"id":2}]} 2024-03-16T10:11:56.777 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 15 2024-03-16T10:11:56.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.772+0000 7f2c56140640 1 -- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2c3c067660 msgr2=0x7f2c3c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.772+0000 7f2c56140640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2c3c067660 0x7f2c3c069b20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2c500b9f60 tx=0x7f2c44007040 comp rx=0 tx=0).stop 2024-03-16T10:11:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.772+0000 7f2c56140640 1 -- 172.21.15.17:0/3556229440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c50147ac0 msgr2=0x7f2c5014de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.772+0000 7f2c56140640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c50147ac0 0x7f2c5014de90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2c40010900 tx=0x7f2c40010dc0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.772+0000 7f2c56140640 1 -- 172.21.15.17:0/3556229440 shutdown_connections 2024-03-16T10:11:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.772+0000 7f2c56140640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2c3c067660 0x7f2c3c069b20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.772+0000 7f2c56140640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c50134f40 0x7f2c501353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.772+0000 7f2c56140640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2c5014d1d0 0x7f2c5014e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.772+0000 7f2c56140640 1 --2- 172.21.15.17:0/3556229440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c50147ac0 0x7f2c5014de90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.772+0000 7f2c56140640 1 -- 172.21.15.17:0/3556229440 >> 172.21.15.17:0/3556229440 conn(0x7f2c500718a0 msgr2=0x7f2c5013fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.772+0000 7f2c56140640 1 -- 172.21.15.17:0/3556229440 shutdown_connections 2024-03-16T10:11:56.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.772+0000 7f2c56140640 1 -- 172.21.15.17:0/3556229440 wait complete. 2024-03-16T10:11:56.795 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:56.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5df239a640 1 Processor -- start 2024-03-16T10:11:56.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5df239a640 1 -- start start 2024-03-16T10:11:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5df239a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5dec0bd760 0x7f5dec1411c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5df239a640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f5dec0b94d0 0x7f5dec141700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5df239a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5dec0bdb40 con 0x7f5dec14abc0 2024-03-16T10:11:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5df239a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f5dec0b4630 con 0x7f5dec0bd760 2024-03-16T10:11:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5df239a640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f5dec0b4910 con 0x7f5dec0b94d0 2024-03-16T10:11:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5debfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5dec0bd760 0x7f5dec1411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5debfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f5dec0bd760 0x7f5dec1411c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55386/0 (socket says 172.21.15.17:55386) 2024-03-16T10:11:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5debfff640 1 -- 172.21.15.17:0/2243835765 learned_addr learned my addr 172.21.15.17:0/2243835765 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5deb7fe640 1 --2- 172.21.15.17:0/2243835765 >> v2:172.21.15.174:3300/0 conn(0x7f5dec0b94d0 0x7f5dec141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5debfff640 1 -- 172.21.15.17:0/2243835765 >> v2:172.21.15.174:3300/0 conn(0x7f5dec0b94d0 msgr2=0x7f5dec141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5debfff640 1 --2- 172.21.15.17:0/2243835765 >> v2:172.21.15.174:3300/0 conn(0x7f5dec0b94d0 0x7f5dec141700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5debfff640 1 -- 172.21.15.17:0/2243835765 >> v1:172.21.15.174:6789/0 conn(0x7f5dec14abc0 legacy=0x7f5dec14b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.916+0000 7f5debfff640 1 -- 172.21.15.17:0/2243835765 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec0b55b0 con 0x7f5dec0bd760 2024-03-16T10:11:56.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5debfff640 1 --2- 172.21.15.17:0/2243835765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec0bd760 0x7f5dec1411c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5dec145010 tx=0x7f5ddc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766c908bc95b2e30 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:56.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5deaffd640 1 -- 172.21.15.17:0/2243835765 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc0b0070 con 0x7f5dec0bd760 2024-03-16T10:11:56.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5deaffd640 1 -- 172.21.15.17:0/2243835765 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ddc0ab600 con 0x7f5dec0bd760 2024-03-16T10:11:56.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5deaffd640 1 -- 172.21.15.17:0/2243835765 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc0ab9d0 con 0x7f5dec0bd760 2024-03-16T10:11:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 -- 172.21.15.17:0/2243835765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec0bd760 msgr2=0x7f5dec1411c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 --2- 172.21.15.17:0/2243835765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec0bd760 0x7f5dec1411c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5dec145010 tx=0x7f5ddc01c3c0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 -- 172.21.15.17:0/2243835765 shutdown_connections 2024-03-16T10:11:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 --2- 172.21.15.17:0/2243835765 >> v2:172.21.15.174:3300/0 conn(0x7f5dec0b94d0 0x7f5dec141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 --2- 172.21.15.17:0/2243835765 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec0bd760 0x7f5dec1411c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 -- 172.21.15.17:0/2243835765 >> 172.21.15.17:0/2243835765 conn(0x7f5dec0b9f20 msgr2=0x7f5dec0bcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 -- 172.21.15.17:0/2243835765 shutdown_connections 2024-03-16T10:11:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 -- 172.21.15.17:0/2243835765 wait complete. 2024-03-16T10:11:56.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 Processor -- start 2024-03-16T10:11:56.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 -- start start 2024-03-16T10:11:56.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec0bd760 0x7f5dec13ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dec14abc0 0x7f5dec1e38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dec1483e0 0x7f5dec1dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec1547d0 con 0x7f5dec0bd760 2024-03-16T10:11:56.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec0b5f50 con 0x7f5dec1483e0 2024-03-16T10:11:56.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f5dec154950 con 0x7f5dec14abc0 2024-03-16T10:11:56.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5debfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec0bd760 0x7f5dec13ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5debfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec0bd760 0x7f5dec13ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55394/0 (socket says 172.21.15.17:55394) 2024-03-16T10:11:56.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5debfff640 1 -- 172.21.15.17:0/3552696486 learned_addr learned my addr 172.21.15.17:0/3552696486 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:56.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df0937640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dec1483e0 0x7f5dec1dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5debfff640 1 -- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dec14abc0 msgr2=0x7f5dec1e38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5deb7fe640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dec14abc0 0x7f5dec1e38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5debfff640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dec14abc0 0x7f5dec1e38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5debfff640 1 -- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dec1483e0 msgr2=0x7f5dec1dffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:56.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5debfff640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dec1483e0 0x7f5dec1dffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:56.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5debfff640 1 -- 172.21.15.17:0/3552696486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec142cb0 con 0x7f5dec0bd760 2024-03-16T10:11:56.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df0937640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dec1483e0 0x7f5dec1dffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:56.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5debfff640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec0bd760 0x7f5dec13ba10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0ae040 tx=0x7f5ddc01ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:56.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5de8ff9640 1 -- 172.21.15.17:0/3552696486 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc0ba020 con 0x7f5dec0bd760 2024-03-16T10:11:56.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5de8ff9640 1 -- 172.21.15.17:0/3552696486 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ddc0b6640 con 0x7f5dec0bd760 2024-03-16T10:11:56.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 -- 172.21.15.17:0/3552696486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec143200 con 0x7f5dec0bd760 2024-03-16T10:11:56.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5de8ff9640 1 -- 172.21.15.17:0/3552696486 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc0b0040 con 0x7f5dec0bd760 2024-03-16T10:11:56.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.920+0000 7f5df239a640 1 -- 172.21.15.17:0/3552696486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec1e33a0 con 0x7f5dec0bd760 2024-03-16T10:11:56.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.924+0000 7f5df239a640 1 -- 172.21.15.17:0/3552696486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec06a130 con 0x7f5dec0bd760 2024-03-16T10:11:56.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.924+0000 7f5de8ff9640 1 -- 172.21.15.17:0/3552696486 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f5ddc0bd020 con 0x7f5dec0bd760 2024-03-16T10:11:56.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.924+0000 7f5de8ff9640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5dd8067660 0x7f5dd8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:56.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.924+0000 7f5de8ff9640 1 -- 172.21.15.17:0/3552696486 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f5ddc11fa80 con 0x7f5dec0bd760 2024-03-16T10:11:56.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.924+0000 7f5deb7fe640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5dd8067660 0x7f5dd8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:56.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.924+0000 7f5deb7fe640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5dd8067660 0x7f5dd8069b20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5dec13ea20 tx=0x7f5de0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:56.928+0000 7f5de8ff9640 1 -- 172.21.15.17:0/3552696486 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5ddc0c4030 con 0x7f5dec0bd760 2024-03-16T10:11:57.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.088+0000 7f5df239a640 1 -- 172.21.15.17:0/3552696486 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5dec142f20 con 0x7f5dec0bd760 2024-03-16T10:11:57.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.091+0000 7f5de8ff9640 1 -- 172.21.15.17:0/3552696486 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16669 (secure 0 0 0) 0x7f5ddc0160d0 con 0x7f5dec0bd760 2024-03-16T10:11:57.097 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:57.097 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":30,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:11:41.019229+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:11:37.284106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T10:11:38.392554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:57.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.091+0000 7f5df239a640 1 -- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5dd8067660 msgr2=0x7f5dd8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:57.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.091+0000 7f5df239a640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5dd8067660 0x7f5dd8069b20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5dec13ea20 tx=0x7f5de0008040 comp rx=0 tx=0).stop 2024-03-16T10:11:57.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.091+0000 7f5df239a640 1 -- 172.21.15.17:0/3552696486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec0bd760 msgr2=0x7f5dec13ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:57.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.091+0000 7f5df239a640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec0bd760 0x7f5dec13ba10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc0ae040 tx=0x7f5ddc01ac00 comp rx=0 tx=0).stop 2024-03-16T10:11:57.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.091+0000 7f5df239a640 1 -- 172.21.15.17:0/3552696486 shutdown_connections 2024-03-16T10:11:57.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.091+0000 7f5df239a640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f5dd8067660 0x7f5dd8069b20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.091+0000 7f5df239a640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dec1483e0 0x7f5dec1dffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.091+0000 7f5df239a640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f5dec14abc0 0x7f5dec1e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.091+0000 7f5df239a640 1 --2- 172.21.15.17:0/3552696486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dec0bd760 0x7f5dec13ba10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.091+0000 7f5df239a640 1 -- 172.21.15.17:0/3552696486 >> 172.21.15.17:0/3552696486 conn(0x7f5dec0b9f20 msgr2=0x7f5dec13ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:57.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.091+0000 7f5df239a640 1 -- 172.21.15.17:0/3552696486 shutdown_connections 2024-03-16T10:11:57.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.091+0000 7f5df239a640 1 -- 172.21.15.17:0/3552696486 wait complete. 2024-03-16T10:11:57.117 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-16T10:11:57.117 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:11:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.227+0000 7f9876cbf640 1 Processor -- start 2024-03-16T10:11:57.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f9876cbf640 1 -- start start 2024-03-16T10:11:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f9876cbf640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f987006fb00 0x7f987009fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f9876cbf640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98700a00c0 0x7f9870147960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f9876cbf640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98700a0570 con 0x7f9870147ea0 2024-03-16T10:11:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f9876cbf640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f9870148350 con 0x7f987006fb00 2024-03-16T10:11:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f9876cbf640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f98700a0850 con 0x7f98700a00c0 2024-03-16T10:11:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f9874a5b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f987006fb00 0x7f987009fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f9874a5b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f987006fb00 0x7f987009fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55410/0 (socket says 172.21.15.17:55410) 2024-03-16T10:11:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f9874a5b640 1 -- 172.21.15.17:0/1754362165 learned_addr learned my addr 172.21.15.17:0/1754362165 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:57.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f986ffff640 1 --2- 172.21.15.17:0/1754362165 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98700a00c0 0x7f9870147960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:57.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f9874a5b640 1 -- 172.21.15.17:0/1754362165 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98700a00c0 msgr2=0x7f9870147960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:57.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f9874a5b640 1 --2- 172.21.15.17:0/1754362165 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98700a00c0 0x7f9870147960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f9874a5b640 1 -- 172.21.15.17:0/1754362165 >> v1:172.21.15.174:6789/0 conn(0x7f9870147ea0 legacy=0x7f98701411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:57.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f9874a5b640 1 -- 172.21.15.17:0/1754362165 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9870147410 con 0x7f987006fb00 2024-03-16T10:11:57.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f9874a5b640 1 --2- 172.21.15.17:0/1754362165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f987006fb00 0x7f987009fb80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9858009870 tx=0x7f985801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f30aec06d8173fa server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:57.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f986f7fe640 1 -- 172.21.15.17:0/1754362165 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98580aa620 con 0x7f987006fb00 2024-03-16T10:11:57.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f986f7fe640 1 -- 172.21.15.17:0/1754362165 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98580af070 con 0x7f987006fb00 2024-03-16T10:11:57.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.231+0000 7f986f7fe640 1 -- 172.21.15.17:0/1754362165 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98580aa9e0 con 0x7f987006fb00 2024-03-16T10:11:57.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 -- 172.21.15.17:0/1754362165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f987006fb00 msgr2=0x7f987009fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:57.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 --2- 172.21.15.17:0/1754362165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f987006fb00 0x7f987009fb80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f9858009870 tx=0x7f985801c560 comp rx=0 tx=0).stop 2024-03-16T10:11:57.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 -- 172.21.15.17:0/1754362165 shutdown_connections 2024-03-16T10:11:57.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 --2- 172.21.15.17:0/1754362165 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f98700a00c0 0x7f9870147960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 --2- 172.21.15.17:0/1754362165 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f987006fb00 0x7f987009fb80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 -- 172.21.15.17:0/1754362165 >> 172.21.15.17:0/1754362165 conn(0x7f9870071910 msgr2=0x7f9870071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 -- 172.21.15.17:0/1754362165 shutdown_connections 2024-03-16T10:11:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 -- 172.21.15.17:0/1754362165 wait complete. 2024-03-16T10:11:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 Processor -- start 2024-03-16T10:11:57.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 -- start start 2024-03-16T10:11:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f987006fb00 0x7f98700b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98700a00c0 0x7f98701e0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9870147ea0 0x7f98701e3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9870154840 con 0x7f98700a00c0 2024-03-16T10:11:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98701546c0 con 0x7f987006fb00 2024-03-16T10:11:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f98701549c0 con 0x7f9870147ea0 2024-03-16T10:11:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f986ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98700a00c0 0x7f98701e0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f986ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98700a00c0 0x7f98701e0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55414/0 (socket says 172.21.15.17:55414) 2024-03-16T10:11:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f986ffff640 1 -- 172.21.15.17:0/3155041060 learned_addr learned my addr 172.21.15.17:0/3155041060 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9874a5b640 1 --2- 172.21.15.17:0/3155041060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f987006fb00 0x7f98700b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:57.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f987525c640 1 --2- 172.21.15.17:0/3155041060 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9870147ea0 0x7f98701e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:57.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f986ffff640 1 -- 172.21.15.17:0/3155041060 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9870147ea0 msgr2=0x7f98701e3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:57.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f986ffff640 1 --2- 172.21.15.17:0/3155041060 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9870147ea0 0x7f98701e3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f986ffff640 1 -- 172.21.15.17:0/3155041060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f987006fb00 msgr2=0x7f98700b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:57.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f986ffff640 1 --2- 172.21.15.17:0/3155041060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f987006fb00 0x7f98700b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f986ffff640 1 -- 172.21.15.17:0/3155041060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98700b4000 con 0x7f98700a00c0 2024-03-16T10:11:57.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f986ffff640 1 --2- 172.21.15.17:0/3155041060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98700a00c0 0x7f98701e0540 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f986000e900 tx=0x7f986000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:57.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f986d7fa640 1 -- 172.21.15.17:0/3155041060 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9860017070 con 0x7f98700a00c0 2024-03-16T10:11:57.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f986d7fa640 1 -- 172.21.15.17:0/3155041060 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f986000ca00 con 0x7f98700a00c0 2024-03-16T10:11:57.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 -- 172.21.15.17:0/3155041060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98700b3890 con 0x7f98700a00c0 2024-03-16T10:11:57.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f986d7fa640 1 -- 172.21.15.17:0/3155041060 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986000cdf0 con 0x7f98700a00c0 2024-03-16T10:11:57.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.235+0000 7f9876cbf640 1 -- 172.21.15.17:0/3155041060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98700affe0 con 0x7f98700a00c0 2024-03-16T10:11:57.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.239+0000 7f9876cbf640 1 -- 172.21.15.17:0/3155041060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98701e0230 con 0x7f98700a00c0 2024-03-16T10:11:57.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.239+0000 7f986d7fa640 1 -- 172.21.15.17:0/3155041060 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f986001a070 con 0x7f98700a00c0 2024-03-16T10:11:57.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.239+0000 7f986d7fa640 1 --2- 172.21.15.17:0/3155041060 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9850067660 0x7f9850069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:57.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.239+0000 7f986d7fa640 1 -- 172.21.15.17:0/3155041060 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f9860086070 con 0x7f98700a00c0 2024-03-16T10:11:57.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.239+0000 7f9874a5b640 1 --2- 172.21.15.17:0/3155041060 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9850067660 0x7f9850069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:57.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.239+0000 7f9874a5b640 1 --2- 172.21.15.17:0/3155041060 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9850067660 0x7f9850069b20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f98580ae000 tx=0x7f98580a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:57.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.243+0000 7f986d7fa640 1 -- 172.21.15.17:0/3155041060 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f98700b4000 con 0x7f98700a00c0 2024-03-16T10:11:57.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.439+0000 7f9876cbf640 1 -- 172.21.15.17:0/3155041060 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98700b0ef0 con 0x7f98700a00c0 2024-03-16T10:11:57.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.439+0000 7f986d7fa640 1 -- 172.21.15.17:0/3155041060 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+3531 (secure 0 0 0) 0x7f98700b0ef0 con 0x7f98700a00c0 2024-03-16T10:11:57.445 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:57.445 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4738,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1674220578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1674220578},{"type":"v1","addr":"172.21.15.174:6835","nonce":1674220578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:11:39.392755+0000","modified":"2024-03-16T10:11:41.419952+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6833/578943207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":578943207},{"type":"v1","addr":"172.21.15.17:6833","nonce":578943207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4953,"qdb_cluster":[4953]},"id":2}]} 2024-03-16T10:11:57.445 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 15 2024-03-16T10:11:57.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.439+0000 7f9876cbf640 1 -- 172.21.15.17:0/3155041060 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9850067660 msgr2=0x7f9850069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:57.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.439+0000 7f9876cbf640 1 --2- 172.21.15.17:0/3155041060 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9850067660 0x7f9850069b20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f98580ae000 tx=0x7f98580a7000 comp rx=0 tx=0).stop 2024-03-16T10:11:57.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.439+0000 7f9876cbf640 1 -- 172.21.15.17:0/3155041060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98700a00c0 msgr2=0x7f98701e0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:57.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.439+0000 7f9876cbf640 1 --2- 172.21.15.17:0/3155041060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98700a00c0 0x7f98701e0540 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f986000e900 tx=0x7f986000edc0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.439+0000 7f9876cbf640 1 -- 172.21.15.17:0/3155041060 shutdown_connections 2024-03-16T10:11:57.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.439+0000 7f9876cbf640 1 --2- 172.21.15.17:0/3155041060 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9850067660 0x7f9850069b20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.439+0000 7f9876cbf640 1 --2- 172.21.15.17:0/3155041060 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9870147ea0 0x7f98701e3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.439+0000 7f9876cbf640 1 --2- 172.21.15.17:0/3155041060 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98700a00c0 0x7f98701e0540 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.439+0000 7f9876cbf640 1 --2- 172.21.15.17:0/3155041060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f987006fb00 0x7f98700b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.439+0000 7f9876cbf640 1 -- 172.21.15.17:0/3155041060 >> 172.21.15.17:0/3155041060 conn(0x7f9870071910 msgr2=0x7f9870145360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:57.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.439+0000 7f9876cbf640 1 -- 172.21.15.17:0/3155041060 shutdown_connections 2024-03-16T10:11:57.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.439+0000 7f9876cbf640 1 -- 172.21.15.17:0/3155041060 wait complete. 2024-03-16T10:11:57.463 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:11:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.575+0000 7f9555362640 1 Processor -- start 2024-03-16T10:11:57.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.575+0000 7f9555362640 1 -- start start 2024-03-16T10:11:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.575+0000 7f9555362640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f95500b72e0 0x7f9550142360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.575+0000 7f9555362640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f955006e660 0x7f95501428a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.575+0000 7f9555362640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f955006f5a0 con 0x7f9550148a30 2024-03-16T10:11:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.575+0000 7f9555362640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f955006fdc0 con 0x7f955006e660 2024-03-16T10:11:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.575+0000 7f9555362640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f95500a0e20 con 0x7f95500b72e0 2024-03-16T10:11:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.575+0000 7f954f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f955006e660 0x7f95501428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.575+0000 7f954f7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f955006e660 0x7f95501428a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55426/0 (socket says 172.21.15.17:55426) 2024-03-16T10:11:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.575+0000 7f954f7fe640 1 -- 172.21.15.17:0/2838514263 learned_addr learned my addr 172.21.15.17:0/2838514263 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.579+0000 7f954e7fc640 1 --2- 172.21.15.17:0/2838514263 >> v2:172.21.15.174:3300/0 conn(0x7f95500b72e0 0x7f9550142360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.579+0000 7f954f7fe640 1 -- 172.21.15.17:0/2838514263 >> v2:172.21.15.174:3300/0 conn(0x7f95500b72e0 msgr2=0x7f9550142360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.579+0000 7f954f7fe640 1 --2- 172.21.15.17:0/2838514263 >> v2:172.21.15.174:3300/0 conn(0x7f95500b72e0 0x7f9550142360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.579+0000 7f954f7fe640 1 -- 172.21.15.17:0/2838514263 >> v1:172.21.15.174:6789/0 conn(0x7f9550148a30 legacy=0x7f9550148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.579+0000 7f954f7fe640 1 -- 172.21.15.17:0/2838514263 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9550149520 con 0x7f955006e660 2024-03-16T10:11:57.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.579+0000 7f954f7fe640 1 --2- 172.21.15.17:0/2838514263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f955006e660 0x7f95501428a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f95500b5440 tx=0x7f954801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21be48009fd9b78e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:57.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.579+0000 7f954dffb640 1 -- 172.21.15.17:0/2838514263 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95480a5660 con 0x7f955006e660 2024-03-16T10:11:57.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.579+0000 7f954dffb640 1 -- 172.21.15.17:0/2838514263 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95480a5800 con 0x7f955006e660 2024-03-16T10:11:57.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.579+0000 7f954dffb640 1 -- 172.21.15.17:0/2838514263 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95480a5bd0 con 0x7f955006e660 2024-03-16T10:11:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 -- 172.21.15.17:0/2838514263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f955006e660 msgr2=0x7f95501428a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 --2- 172.21.15.17:0/2838514263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f955006e660 0x7f95501428a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f95500b5440 tx=0x7f954801c890 comp rx=0 tx=0).stop 2024-03-16T10:11:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 -- 172.21.15.17:0/2838514263 shutdown_connections 2024-03-16T10:11:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 --2- 172.21.15.17:0/2838514263 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f955006e660 0x7f95501428a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 --2- 172.21.15.17:0/2838514263 >> v2:172.21.15.174:3300/0 conn(0x7f95500b72e0 0x7f9550142360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 -- 172.21.15.17:0/2838514263 >> 172.21.15.17:0/2838514263 conn(0x7f95500718a0 msgr2=0x7f9550071cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:57.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 -- 172.21.15.17:0/2838514263 shutdown_connections 2024-03-16T10:11:57.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 -- 172.21.15.17:0/2838514263 wait complete. 2024-03-16T10:11:57.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 Processor -- start 2024-03-16T10:11:57.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 -- start start 2024-03-16T10:11:57.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f955006e660 0x7f955013b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:57.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9550148a30 0x7f95501df630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:57.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9550139760 0x7f955013e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:57.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9550152530 con 0x7f9550139760 2024-03-16T10:11:57.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95501496f0 con 0x7f9550148a30 2024-03-16T10:11:57.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f95501526b0 con 0x7f955006e660 2024-03-16T10:11:57.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f954f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9550139760 0x7f955013e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:57.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f954f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9550139760 0x7f955013e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55438/0 (socket says 172.21.15.17:55438) 2024-03-16T10:11:57.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f954f7fe640 1 -- 172.21.15.17:0/36046089 learned_addr learned my addr 172.21.15.17:0/36046089 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f954e7fc640 1 --2- 172.21.15.17:0/36046089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9550148a30 0x7f95501df630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f954effd640 1 --2- 172.21.15.17:0/36046089 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f955006e660 0x7f955013b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f954f7fe640 1 -- 172.21.15.17:0/36046089 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f955006e660 msgr2=0x7f955013b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f954f7fe640 1 --2- 172.21.15.17:0/36046089 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f955006e660 0x7f955013b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f954f7fe640 1 -- 172.21.15.17:0/36046089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9550148a30 msgr2=0x7f95501df630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f954f7fe640 1 --2- 172.21.15.17:0/36046089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9550148a30 0x7f95501df630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f954f7fe640 1 -- 172.21.15.17:0/36046089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955013a780 con 0x7f9550139760 2024-03-16T10:11:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f954f7fe640 1 --2- 172.21.15.17:0/36046089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9550139760 0x7f955013e870 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f954801a4c0 tx=0x7f9548019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f95477fe640 1 -- 172.21.15.17:0/36046089 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95480bc020 con 0x7f9550139760 2024-03-16T10:11:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f95477fe640 1 -- 172.21.15.17:0/36046089 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9548007bb0 con 0x7f9550139760 2024-03-16T10:11:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f95477fe640 1 -- 172.21.15.17:0/36046089 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95480a5660 con 0x7f9550139760 2024-03-16T10:11:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 -- 172.21.15.17:0/36046089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955013aa60 con 0x7f9550139760 2024-03-16T10:11:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.583+0000 7f9555362640 1 -- 172.21.15.17:0/36046089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955013a3d0 con 0x7f9550139760 2024-03-16T10:11:57.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.587+0000 7f9555362640 1 -- 172.21.15.17:0/36046089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f955006e3a0 con 0x7f9550139760 2024-03-16T10:11:57.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.587+0000 7f95477fe640 1 -- 172.21.15.17:0/36046089 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f95480bf020 con 0x7f9550139760 2024-03-16T10:11:57.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.587+0000 7f95477fe640 1 --2- 172.21.15.17:0/36046089 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9530067660 0x7f9530069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:57.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.587+0000 7f95477fe640 1 -- 172.21.15.17:0/36046089 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f9548120aa0 con 0x7f9550139760 2024-03-16T10:11:57.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.587+0000 7f954effd640 1 --2- 172.21.15.17:0/36046089 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9530067660 0x7f9530069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:57.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.587+0000 7f954effd640 1 --2- 172.21.15.17:0/36046089 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9530067660 0x7f9530069b20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f955013bf50 tx=0x7f9540008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.591+0000 7f95477fe640 1 -- 172.21.15.17:0/36046089 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f95480c6030 con 0x7f9550139760 2024-03-16T10:11:57.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.991+0000 7f9555362640 1 -- 172.21.15.17:0/36046089 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f955006a130 con 0x7f9550139760 2024-03-16T10:11:57.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.991+0000 7f95477fe640 1 -- 172.21.15.17:0/36046089 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+3531 (secure 0 0 0) 0x7f95480165a0 con 0x7f9550139760 2024-03-16T10:11:57.999 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:57.999 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4738,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1674220578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1674220578},{"type":"v1","addr":"172.21.15.174:6835","nonce":1674220578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T10:11:39.392755+0000","modified":"2024-03-16T10:11:41.419952+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4953},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4953":{"gid":4953,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.17:6833/578943207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":578943207},{"type":"v1","addr":"172.21.15.17:6833","nonce":578943207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4953,"qdb_cluster":[4953]},"id":2}]} 2024-03-16T10:11:58.000 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 15 2024-03-16T10:11:58.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.995+0000 7f9555362640 1 -- 172.21.15.17:0/36046089 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9530067660 msgr2=0x7f9530069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.995+0000 7f9555362640 1 --2- 172.21.15.17:0/36046089 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9530067660 0x7f9530069b20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f955013bf50 tx=0x7f9540008040 comp rx=0 tx=0).stop 2024-03-16T10:11:58.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.995+0000 7f9555362640 1 -- 172.21.15.17:0/36046089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9550139760 msgr2=0x7f955013e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.995+0000 7f9555362640 1 --2- 172.21.15.17:0/36046089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9550139760 0x7f955013e870 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f954801a4c0 tx=0x7f9548019630 comp rx=0 tx=0).stop 2024-03-16T10:11:58.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.995+0000 7f9555362640 1 -- 172.21.15.17:0/36046089 shutdown_connections 2024-03-16T10:11:58.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.995+0000 7f9555362640 1 --2- 172.21.15.17:0/36046089 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9530067660 0x7f9530069b20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.995+0000 7f9555362640 1 --2- 172.21.15.17:0/36046089 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9550139760 0x7f955013e870 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.995+0000 7f9555362640 1 --2- 172.21.15.17:0/36046089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9550148a30 0x7f95501df630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.995+0000 7f9555362640 1 --2- 172.21.15.17:0/36046089 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f955006e660 0x7f955013b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.995+0000 7f9555362640 1 -- 172.21.15.17:0/36046089 >> 172.21.15.17:0/36046089 conn(0x7f95500718a0 msgr2=0x7f955013fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:58.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.995+0000 7f9555362640 1 -- 172.21.15.17:0/36046089 shutdown_connections 2024-03-16T10:11:58.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:57.995+0000 7f9555362640 1 -- 172.21.15.17:0/36046089 wait complete. 2024-03-16T10:11:58.018 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:11:58.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.123+0000 7fe5b7ee7640 1 Processor -- start 2024-03-16T10:11:58.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.123+0000 7fe5b7ee7640 1 -- start start 2024-03-16T10:11:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.123+0000 7fe5b7ee7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5b0067220 0x7fe5b014aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.123+0000 7fe5b7ee7640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5b0067600 con 0x7fe5b0147fa0 2024-03-16T10:11:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.123+0000 7fe5b7ee7640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5b00b41c0 con 0x7fe5b00b3ca0 2024-03-16T10:11:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.123+0000 7fe5b7ee7640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe5b0148410 con 0x7fe5b0067220 2024-03-16T10:11:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b6484640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fe5b0147fa0 0x7fe5b0151d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:42938/0 (socket says 172.21.15.17:42938) 2024-03-16T10:11:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b6484640 1 -- 172.21.15.17:0/1327539142 learned_addr learned my addr 172.21.15.17:0/1327539142 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:11:58.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b5c83640 1 --2- 172.21.15.17:0/1327539142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5b0067220 0x7fe5b014aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 -- 172.21.15.17:0/1327539142 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1409920613 0 0) 0x7fe5b0067600 con 0x7fe5b0147fa0 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 -- 172.21.15.17:0/1327539142 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe59c003530 con 0x7fe5b0147fa0 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 -- 172.21.15.17:0/1327539142 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1406816591 0 0) 0x7fe59c003530 con 0x7fe5b0147fa0 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 -- 172.21.15.17:0/1327539142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5b0067220 msgr2=0x7fe5b014aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 --2- 172.21.15.17:0/1327539142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5b0067220 0x7fe5b014aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 -- 172.21.15.17:0/1327539142 >> v1:172.21.15.174:6789/0 conn(0x7fe5b00b3ca0 legacy=0x7fe5b0147890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 -- 172.21.15.17:0/1327539142 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b00b44a0 con 0x7fe5b0147fa0 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 -- 172.21.15.17:0/1327539142 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fe5a8001a00 con 0x7fe5b0147fa0 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 -- 172.21.15.17:0/1327539142 >> v1:172.21.15.17:6789/0 conn(0x7fe5b0147fa0 legacy=0x7fe5b0151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 --2- 172.21.15.17:0/1327539142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5b0067220 0x7fe59c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 --2- 172.21.15.17:0/1327539142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe59c003fc0 0x7fe59c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 --2- 172.21.15.17:0/1327539142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c004d50 0x7fe59c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 -- 172.21.15.17:0/1327539142 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c00bb90 con 0x7fe59c003fc0 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 -- 172.21.15.17:0/1327539142 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe59c00bd40 con 0x7fe59c004d50 2024-03-16T10:11:58.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 -- 172.21.15.17:0/1327539142 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe59c00bf40 con 0x7fe5b0067220 2024-03-16T10:11:58.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b5482640 1 --2- 172.21.15.17:0/1327539142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe59c003fc0 0x7fe59c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b6484640 1 --2- 172.21.15.17:0/1327539142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c004d50 0x7fe59c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b5c83640 1 --2- 172.21.15.17:0/1327539142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5b0067220 0x7fe59c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b5482640 1 -- 172.21.15.17:0/1327539142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5b0067220 msgr2=0x7fe59c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:58.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b5482640 1 --2- 172.21.15.17:0/1327539142 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5b0067220 0x7fe59c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b5482640 1 -- 172.21.15.17:0/1327539142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c004d50 msgr2=0x7fe59c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:58.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b5482640 1 --2- 172.21.15.17:0/1327539142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c004d50 0x7fe59c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b5482640 1 -- 172.21.15.17:0/1327539142 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe59c003530 con 0x7fe59c003fc0 2024-03-16T10:11:58.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b6484640 1 --2- 172.21.15.17:0/1327539142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe59c004d50 0x7fe59c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:58.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b5482640 1 --2- 172.21.15.17:0/1327539142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe59c003fc0 0x7fe59c004810 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0001780 tx=0x7fe5a001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd35f7cbcf6adad6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:58.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 -- 172.21.15.17:0/1327539142 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00013a0 con 0x7fe59c003fc0 2024-03-16T10:11:58.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 -- 172.21.15.17:0/1327539142 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a00052f0 con 0x7fe59c003fc0 2024-03-16T10:11:58.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b4c81640 1 -- 172.21.15.17:0/1327539142 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0003840 con 0x7fe59c003fc0 2024-03-16T10:11:58.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1327539142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe59c003fc0 msgr2=0x7fe59c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b7ee7640 1 --2- 172.21.15.17:0/1327539142 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe59c003fc0 0x7fe59c004810 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0001780 tx=0x7fe5a001c420 comp rx=0 tx=0).stop 2024-03-16T10:11:58.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b6484640 1 -- 172.21.15.17:0/1327539142 reap_dead start 2024-03-16T10:11:58.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1327539142 shutdown_connections 2024-03-16T10:11:58.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.127+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1327539142 >> 172.21.15.17:0/1327539142 conn(0x7fe5b006be20 msgr2=0x7fe5b00b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:58.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1327539142 shutdown_connections 2024-03-16T10:11:58.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1327539142 wait complete. 2024-03-16T10:11:58.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b7ee7640 1 Processor -- start 2024-03-16T10:11:58.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b7ee7640 1 -- start start 2024-03-16T10:11:58.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b7ee7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5b014e890 0x7fe5b01dfb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b7ee7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b01e0070 0x7fe5b0151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b7ee7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5b0152010 0x7fe5b01e30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b7ee7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01535e0 con 0x7fe5b01e0070 2024-03-16T10:11:58.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b7ee7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b0153460 con 0x7fe5b0152010 2024-03-16T10:11:58.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b7ee7640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fe5b0153760 con 0x7fe5b014e890 2024-03-16T10:11:58.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b5482640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b01e0070 0x7fe5b0151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b5482640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b01e0070 0x7fe5b0151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55460/0 (socket says 172.21.15.17:55460) 2024-03-16T10:11:58.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b5482640 1 -- 172.21.15.17:0/1826675362 learned_addr learned my addr 172.21.15.17:0/1826675362 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:58.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b5c83640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5b014e890 0x7fe5b01dfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b6484640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5b0152010 0x7fe5b01e30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b5482640 1 -- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5b014e890 msgr2=0x7fe5b01dfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b5482640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5b014e890 0x7fe5b01dfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b5482640 1 -- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5b0152010 msgr2=0x7fe5b01e30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b5482640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5b0152010 0x7fe5b01e30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b5482640 1 -- 172.21.15.17:0/1826675362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b0150330 con 0x7fe5b01e0070 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b6484640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5b0152010 0x7fe5b01e30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b5482640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b01e0070 0x7fe5b0151ad0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0000c10 tx=0x7fe59c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5a67fc640 1 -- 172.21.15.17:0/1826675362 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a00b8020 con 0x7fe5b01e0070 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5a67fc640 1 -- 172.21.15.17:0/1826675362 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5a00054a0 con 0x7fe5b01e0070 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1826675362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b014e2e0 con 0x7fe5b01e0070 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5a67fc640 1 -- 172.21.15.17:0/1826675362 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0007a40 con 0x7fe5b01e0070 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1826675362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b014ec70 con 0x7fe5b01e0070 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1826675362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5b006a130 con 0x7fe5b01e0070 2024-03-16T10:11:58.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.131+0000 7fe5a67fc640 1 -- 172.21.15.17:0/1826675362 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe5a00bb020 con 0x7fe5b01e0070 2024-03-16T10:11:58.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.135+0000 7fe5a67fc640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe594067660 0x7fe594069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.135+0000 7fe5a67fc640 1 -- 172.21.15.17:0/1826675362 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fe5a011f740 con 0x7fe5b01e0070 2024-03-16T10:11:58.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.135+0000 7fe5b5c83640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe594067660 0x7fe594069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.135+0000 7fe5b5c83640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe594067660 0x7fe594069b20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009a30 tx=0x7fe5ac008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:58.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.139+0000 7fe5a67fc640 1 -- 172.21.15.17:0/1826675362 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe5a00c2030 con 0x7fe5b01e0070 2024-03-16T10:11:58.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.295+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1826675362 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5b0148ed0 con 0x7fe5b01e0070 2024-03-16T10:11:58.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.295+0000 7fe5a67fc640 1 -- 172.21.15.17:0/1826675362 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16669 (secure 0 0 0) 0x7fe5a00a4e20 con 0x7fe5b01e0070 2024-03-16T10:11:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:11:58.303 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":30,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:11:41.019229+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T10:11:37.284106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T10:11:38.392554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:11:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.299+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe594067660 msgr2=0x7fe594069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.299+0000 7fe5b7ee7640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe594067660 0x7fe594069b20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac009a30 tx=0x7fe5ac008040 comp rx=0 tx=0).stop 2024-03-16T10:11:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.299+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1826675362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b01e0070 msgr2=0x7fe5b0151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.299+0000 7fe5b7ee7640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b01e0070 0x7fe5b0151ad0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0000c10 tx=0x7fe59c005840 comp rx=0 tx=0).stop 2024-03-16T10:11:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.299+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1826675362 shutdown_connections 2024-03-16T10:11:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.299+0000 7fe5b7ee7640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fe594067660 0x7fe594069b20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.299+0000 7fe5b7ee7640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5b0152010 0x7fe5b01e30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.299+0000 7fe5b7ee7640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe5b01e0070 0x7fe5b0151ad0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.299+0000 7fe5b7ee7640 1 --2- 172.21.15.17:0/1826675362 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fe5b014e890 0x7fe5b01dfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.299+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1826675362 >> 172.21.15.17:0/1826675362 conn(0x7fe5b006be20 msgr2=0x7fe5b00b5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:58.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.299+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1826675362 shutdown_connections 2024-03-16T10:11:58.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.299+0000 7fe5b7ee7640 1 -- 172.21.15.17:0/1826675362 wait complete. 2024-03-16T10:11:58.321 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-16T10:11:58.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa31ad0e640 1 Processor -- start 2024-03-16T10:11:58.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa31ad0e640 1 -- start start 2024-03-16T10:11:58.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa31ad0e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa3140b60e0 0x7fa31413e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa31ad0e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa3140aed90 0x7fa31414ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa31ad0e640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa31406dbf0 con 0x7fa31406d810 2024-03-16T10:11:58.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa31ad0e640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa3140b0190 con 0x7fa3140aed90 2024-03-16T10:11:58.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa31ad0e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa3140b5b10 con 0x7fa3140b60e0 2024-03-16T10:11:58.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa3192ab640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa3140aed90 0x7fa31414ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa3192ab640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa3140aed90 0x7fa31414ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55464/0 (socket says 172.21.15.17:55464) 2024-03-16T10:11:58.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa3192ab640 1 -- 172.21.15.17:0/1007885218 learned_addr learned my addr 172.21.15.17:0/1007885218 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:58.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa30bfff640 1 --2- 172.21.15.17:0/1007885218 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa3140b60e0 0x7fa31413e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa3192ab640 1 -- 172.21.15.17:0/1007885218 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa3140b60e0 msgr2=0x7fa31413e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:58.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa3192ab640 1 --2- 172.21.15.17:0/1007885218 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa3140b60e0 0x7fa31413e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa3192ab640 1 -- 172.21.15.17:0/1007885218 >> v1:172.21.15.174:6789/0 conn(0x7fa31406d810 legacy=0x7fa3140af330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.431+0000 7fa3192ab640 1 -- 172.21.15.17:0/1007885218 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3140afc70 con 0x7fa3140aed90 2024-03-16T10:11:58.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa3192ab640 1 --2- 172.21.15.17:0/1007885218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3140aed90 0x7fa31414ea70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009e50 tx=0x7fa30c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84b4b914d731d62 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:58.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa30b7fe640 1 -- 172.21.15.17:0/1007885218 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0a5700 con 0x7fa3140aed90 2024-03-16T10:11:58.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa30b7fe640 1 -- 172.21.15.17:0/1007885218 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30c0a58a0 con 0x7fa3140aed90 2024-03-16T10:11:58.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa30b7fe640 1 -- 172.21.15.17:0/1007885218 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0a5c70 con 0x7fa3140aed90 2024-03-16T10:11:58.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 -- 172.21.15.17:0/1007885218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3140aed90 msgr2=0x7fa31414ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 --2- 172.21.15.17:0/1007885218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3140aed90 0x7fa31414ea70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009e50 tx=0x7fa30c01c8e0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 -- 172.21.15.17:0/1007885218 shutdown_connections 2024-03-16T10:11:58.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 --2- 172.21.15.17:0/1007885218 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3140aed90 0x7fa31414ea70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 --2- 172.21.15.17:0/1007885218 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa3140b60e0 0x7fa31413e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 -- 172.21.15.17:0/1007885218 >> 172.21.15.17:0/1007885218 conn(0x7fa3140b9a30 msgr2=0x7fa3140b9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:58.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 -- 172.21.15.17:0/1007885218 shutdown_connections 2024-03-16T10:11:58.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 -- 172.21.15.17:0/1007885218 wait complete. 2024-03-16T10:11:58.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 Processor -- start 2024-03-16T10:11:58.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 -- start start 2024-03-16T10:11:58.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa31406d810 0x7fa31409f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa3140aed90 0x7fa314140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3140b60e0 0x7fa31413fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa314151b60 con 0x7fa31406d810 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3141519e0 con 0x7fa3140b60e0 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa31ad0e640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa314151ce0 con 0x7fa3140aed90 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa318aaa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa31406d810 0x7fa31409f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa318aaa640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa31406d810 0x7fa31409f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55476/0 (socket says 172.21.15.17:55476) 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa318aaa640 1 -- 172.21.15.17:0/701041926 learned_addr learned my addr 172.21.15.17:0/701041926 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa3192ab640 1 --2- 172.21.15.17:0/701041926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3140b60e0 0x7fa31413fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa30bfff640 1 --2- 172.21.15.17:0/701041926 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa3140aed90 0x7fa314140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa318aaa640 1 -- 172.21.15.17:0/701041926 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa3140aed90 msgr2=0x7fa314140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa318aaa640 1 --2- 172.21.15.17:0/701041926 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa3140aed90 0x7fa314140480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa318aaa640 1 -- 172.21.15.17:0/701041926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3140b60e0 msgr2=0x7fa31413fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa318aaa640 1 --2- 172.21.15.17:0/701041926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3140b60e0 0x7fa31413fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.435+0000 7fa318aaa640 1 -- 172.21.15.17:0/701041926 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3141381b0 con 0x7fa31406d810 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.439+0000 7fa318aaa640 1 --2- 172.21.15.17:0/701041926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa31406d810 0x7fa31409f0b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c500 tx=0x7fa31000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.439+0000 7fa3097fa640 1 -- 172.21.15.17:0/701041926 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310016020 con 0x7fa31406d810 2024-03-16T10:11:58.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.439+0000 7fa3097fa640 1 -- 172.21.15.17:0/701041926 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa310005220 con 0x7fa31406d810 2024-03-16T10:11:58.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.439+0000 7fa31ad0e640 1 -- 172.21.15.17:0/701041926 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3141393c0 con 0x7fa31406d810 2024-03-16T10:11:58.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.439+0000 7fa3097fa640 1 -- 172.21.15.17:0/701041926 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310007030 con 0x7fa31406d810 2024-03-16T10:11:58.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.439+0000 7fa31ad0e640 1 -- 172.21.15.17:0/701041926 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa314138450 con 0x7fa31406d810 2024-03-16T10:11:58.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.439+0000 7fa31ad0e640 1 -- 172.21.15.17:0/701041926 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa31406a130 con 0x7fa31406d810 2024-03-16T10:11:58.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.439+0000 7fa3097fa640 1 -- 172.21.15.17:0/701041926 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa3100071d0 con 0x7fa31406d810 2024-03-16T10:11:58.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.439+0000 7fa3097fa640 1 --2- 172.21.15.17:0/701041926 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa300067660 0x7fa300069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.439+0000 7fa3097fa640 1 -- 172.21.15.17:0/701041926 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fa3100855d0 con 0x7fa31406d810 2024-03-16T10:11:58.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.439+0000 7fa30bfff640 1 --2- 172.21.15.17:0/701041926 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa300067660 0x7fa300069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.443+0000 7fa30bfff640 1 --2- 172.21.15.17:0/701041926 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa300067660 0x7fa300069b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa314137210 tx=0x7fa304008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:58.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.443+0000 7fa3097fa640 1 -- 172.21.15.17:0/701041926 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa310056240 con 0x7fa31406d810 2024-03-16T10:11:58.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.631+0000 7fa31ad0e640 1 -- 172.21.15.17:0/701041926 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa314071660 con 0x7fa31406d810 2024-03-16T10:11:58.640 INFO:tasks.ceph.mon.a.smithi017.stderr:2024-03-16T10:11:58.635+0000 7f381f0a6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T10:11:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.635+0000 7fa3097fa640 1 -- 172.21.15.17:0/701041926 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) ==== 161+0+0 (secure 0 0 0) 0x7fa310055b40 con 0x7fa31406d810 2024-03-16T10:11:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-16T10:11:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.639+0000 7fa31ad0e640 1 -- 172.21.15.17:0/701041926 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa300067660 msgr2=0x7fa300069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.639+0000 7fa31ad0e640 1 --2- 172.21.15.17:0/701041926 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa300067660 0x7fa300069b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa314137210 tx=0x7fa304008040 comp rx=0 tx=0).stop 2024-03-16T10:11:58.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.639+0000 7fa31ad0e640 1 -- 172.21.15.17:0/701041926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa31406d810 msgr2=0x7fa31409f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.639+0000 7fa31ad0e640 1 --2- 172.21.15.17:0/701041926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa31406d810 0x7fa31409f0b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c500 tx=0x7fa31000c9c0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.639+0000 7fa31ad0e640 1 -- 172.21.15.17:0/701041926 shutdown_connections 2024-03-16T10:11:58.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.639+0000 7fa31ad0e640 1 --2- 172.21.15.17:0/701041926 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa300067660 0x7fa300069b20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.639+0000 7fa31ad0e640 1 --2- 172.21.15.17:0/701041926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3140b60e0 0x7fa31413fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.639+0000 7fa31ad0e640 1 --2- 172.21.15.17:0/701041926 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa3140aed90 0x7fa314140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.639+0000 7fa31ad0e640 1 --2- 172.21.15.17:0/701041926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa31406d810 0x7fa31409f0b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.639+0000 7fa31ad0e640 1 -- 172.21.15.17:0/701041926 >> 172.21.15.17:0/701041926 conn(0x7fa3140b9a30 msgr2=0x7fa3140b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:58.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.639+0000 7fa31ad0e640 1 -- 172.21.15.17:0/701041926 shutdown_connections 2024-03-16T10:11:58.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.639+0000 7fa31ad0e640 1 -- 172.21.15.17:0/701041926 wait complete. 2024-03-16T10:11:58.647 INFO:tasks.ceph.mds.a.smithi017.stderr: -2783> 2024-03-16T10:11:43.328+0000 7f3ac45d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-16T10:11:58.662 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-16T10:11:58.675 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-03-16T10:11:58.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a7054b640 1 Processor -- start 2024-03-16T10:11:58.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a7054b640 1 -- start start 2024-03-16T10:11:58.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a7054b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2a680b2400 0x7f2a681452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a7054b640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f2a680b4200 0x7f2a6814fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a7054b640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a680af0c0 con 0x7f2a680ae060 2024-03-16T10:11:58.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a7054b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f2a680afcf0 con 0x7f2a680b2400 2024-03-16T10:11:58.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a7054b640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f2a680b3fe0 con 0x7f2a680b4200 2024-03-16T10:11:58.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a6e2e7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2a680b2400 0x7f2a681452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a6e2e7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f2a680b2400 0x7f2a681452c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55498/0 (socket says 172.21.15.17:55498) 2024-03-16T10:11:58.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a6e2e7640 1 -- 172.21.15.17:0/2232505285 learned_addr learned my addr 172.21.15.17:0/2232505285 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:58.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a6e2e7640 1 -- 172.21.15.17:0/2232505285 >> v2:172.21.15.174:3300/0 conn(0x7f2a680b4200 msgr2=0x7f2a6814fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:58.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a6e2e7640 1 --2- 172.21.15.17:0/2232505285 >> v2:172.21.15.174:3300/0 conn(0x7f2a680b4200 0x7f2a6814fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a6e2e7640 1 -- 172.21.15.17:0/2232505285 >> v1:172.21.15.174:6789/0 conn(0x7f2a680ae060 legacy=0x7f2a68072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:58.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a6e2e7640 1 -- 172.21.15.17:0/2232505285 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a680b27e0 con 0x7f2a680b2400 2024-03-16T10:11:58.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.771+0000 7f2a6e2e7640 1 --2- 172.21.15.17:0/2232505285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a680b2400 0x7f2a681452c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2a680b3bd0 tx=0x7f2a6001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a375a4d4f35d596e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:58.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6d2e5640 1 -- 172.21.15.17:0/2232505285 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600a5810 con 0x7f2a680b2400 2024-03-16T10:11:58.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6d2e5640 1 -- 172.21.15.17:0/2232505285 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a600a59b0 con 0x7f2a680b2400 2024-03-16T10:11:58.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6d2e5640 1 -- 172.21.15.17:0/2232505285 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600a5d80 con 0x7f2a680b2400 2024-03-16T10:11:58.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 -- 172.21.15.17:0/2232505285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a680b2400 msgr2=0x7f2a681452c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 --2- 172.21.15.17:0/2232505285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a680b2400 0x7f2a681452c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2a680b3bd0 tx=0x7f2a6001c9b0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 -- 172.21.15.17:0/2232505285 shutdown_connections 2024-03-16T10:11:58.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 --2- 172.21.15.17:0/2232505285 >> v2:172.21.15.174:3300/0 conn(0x7f2a680b4200 0x7f2a6814fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 --2- 172.21.15.17:0/2232505285 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a680b2400 0x7f2a681452c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 -- 172.21.15.17:0/2232505285 >> 172.21.15.17:0/2232505285 conn(0x7f2a680b8e90 msgr2=0x7f2a680b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:58.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 -- 172.21.15.17:0/2232505285 shutdown_connections 2024-03-16T10:11:58.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 -- 172.21.15.17:0/2232505285 wait complete. 2024-03-16T10:11:58.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 Processor -- start 2024-03-16T10:11:58.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 -- start start 2024-03-16T10:11:58.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a680ae060 0x7f2a68133dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a680b4200 0x7f2a68134d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a6813f410 0x7f2a68135290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2a68145980 con 0x7f2a680b4200 2024-03-16T10:11:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a68145800 con 0x7f2a6813f410 2024-03-16T10:11:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2a68073290 con 0x7f2a680ae060 2024-03-16T10:11:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6dae6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a680b4200 0x7f2a68134d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6dae6640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a680b4200 0x7f2a68134d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55504/0 (socket says 172.21.15.17:55504) 2024-03-16T10:11:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6dae6640 1 -- 172.21.15.17:0/442400850 learned_addr learned my addr 172.21.15.17:0/442400850 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6e2e7640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a680ae060 0x7f2a68133dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6eae8640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a6813f410 0x7f2a68135290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6dae6640 1 -- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a680ae060 msgr2=0x7f2a68133dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6dae6640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a680ae060 0x7f2a68133dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6dae6640 1 -- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a6813f410 msgr2=0x7f2a68135290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:58.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6dae6640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a6813f410 0x7f2a68135290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6dae6640 1 -- 172.21.15.17:0/442400850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a68072840 con 0x7f2a680b4200 2024-03-16T10:11:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6eae8640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a6813f410 0x7f2a68135290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6e2e7640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a680ae060 0x7f2a68133dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:11:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a6dae6640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a680b4200 0x7f2a68134d50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800c490 tx=0x7f2a58001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a5effd640 1 -- 172.21.15.17:0/442400850 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58017070 con 0x7f2a680b4200 2024-03-16T10:11:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a5effd640 1 -- 172.21.15.17:0/442400850 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a58003600 con 0x7f2a680b4200 2024-03-16T10:11:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 -- 172.21.15.17:0/442400850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a68134530 con 0x7f2a680b4200 2024-03-16T10:11:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a5effd640 1 -- 172.21.15.17:0/442400850 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58003a30 con 0x7f2a680b4200 2024-03-16T10:11:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.775+0000 7f2a7054b640 1 -- 172.21.15.17:0/442400850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a68138490 con 0x7f2a680b4200 2024-03-16T10:11:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.779+0000 7f2a7054b640 1 -- 172.21.15.17:0/442400850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a68134a40 con 0x7f2a680b4200 2024-03-16T10:11:58.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.779+0000 7f2a5effd640 1 -- 172.21.15.17:0/442400850 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f2a5801a070 con 0x7f2a680b4200 2024-03-16T10:11:58.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.779+0000 7f2a5effd640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2a4c067660 0x7f2a4c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:58.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.779+0000 7f2a5effd640 1 -- 172.21.15.17:0/442400850 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5795+0+0 (secure 0 0 0) 0x7f2a58084d80 con 0x7f2a680b4200 2024-03-16T10:11:58.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.779+0000 7f2a6e2e7640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2a4c067660 0x7f2a4c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:58.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.779+0000 7f2a6e2e7640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2a4c067660 0x7f2a4c069b20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2a600ae000 tx=0x7f2a600a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.783+0000 7f2a5effd640 1 -- 172.21.15.17:0/442400850 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2a58055990 con 0x7f2a680b4200 2024-03-16T10:11:58.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:58.979+0000 7f2a7054b640 1 -- 172.21.15.17:0/442400850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2a68068360 con 0x7f2a680b4200 2024-03-16T10:11:59.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.639+0000 7f2a5effd640 1 -- 172.21.15.17:0/442400850 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f2a58055290 con 0x7f2a680b4200 2024-03-16T10:11:59.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.643+0000 7f2a7054b640 1 -- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2a4c067660 msgr2=0x7f2a4c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:59.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.643+0000 7f2a7054b640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2a4c067660 0x7f2a4c069b20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2a600ae000 tx=0x7f2a600a7070 comp rx=0 tx=0).stop 2024-03-16T10:11:59.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.643+0000 7f2a7054b640 1 -- 172.21.15.17:0/442400850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a680b4200 msgr2=0x7f2a68134d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:59.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.643+0000 7f2a7054b640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a680b4200 0x7f2a68134d50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800c490 tx=0x7f2a58001e80 comp rx=0 tx=0).stop 2024-03-16T10:11:59.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.643+0000 7f2a7054b640 1 -- 172.21.15.17:0/442400850 shutdown_connections 2024-03-16T10:11:59.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.643+0000 7f2a7054b640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2a4c067660 0x7f2a4c069b20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:59.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.643+0000 7f2a7054b640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a6813f410 0x7f2a68135290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:59.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.643+0000 7f2a7054b640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2a680b4200 0x7f2a68134d50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:59.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.643+0000 7f2a7054b640 1 --2- 172.21.15.17:0/442400850 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2a680ae060 0x7f2a68133dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:59.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.643+0000 7f2a7054b640 1 -- 172.21.15.17:0/442400850 >> 172.21.15.17:0/442400850 conn(0x7f2a680b8e90 msgr2=0x7f2a6806c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:59.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.643+0000 7f2a7054b640 1 -- 172.21.15.17:0/442400850 shutdown_connections 2024-03-16T10:11:59.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.643+0000 7f2a7054b640 1 -- 172.21.15.17:0/442400850 wait complete. 2024-03-16T10:11:59.672 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-16T10:11:59.853 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T10:11:59.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f8784a18640 1 Processor -- start 2024-03-16T10:11:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f8784a18640 1 -- start start 2024-03-16T10:11:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f8784a18640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f87800b2400 0x7f87801452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f8784a18640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f87800ae060 0x7f8780072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f8784a18640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87800af0c0 con 0x7f87800730c0 2024-03-16T10:11:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f8784a18640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f87800afcf0 con 0x7f87800ae060 2024-03-16T10:11:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f8784a18640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f87800b3fe0 con 0x7f87800b2400 2024-03-16T10:11:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f877dd9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f87800ae060 0x7f8780072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f877dd9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f87800ae060 0x7f8780072b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55314/0 (socket says 172.21.15.17:55314) 2024-03-16T10:11:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f877dd9b640 1 -- 172.21.15.17:0/3210410528 learned_addr learned my addr 172.21.15.17:0/3210410528 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f877e59c640 1 --2- 172.21.15.17:0/3210410528 >> v2:172.21.15.174:3300/0 conn(0x7f87800b2400 0x7f87801452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f877dd9b640 1 -- 172.21.15.17:0/3210410528 >> v2:172.21.15.174:3300/0 conn(0x7f87800b2400 msgr2=0x7f87801452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:11:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f877dd9b640 1 --2- 172.21.15.17:0/3210410528 >> v2:172.21.15.174:3300/0 conn(0x7f87800b2400 0x7f87801452c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f877dd9b640 1 -- 172.21.15.17:0/3210410528 >> v1:172.21.15.174:6789/0 conn(0x7f87800730c0 legacy=0x7f878006c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:59.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.847+0000 7f877dd9b640 1 -- 172.21.15.17:0/3210410528 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87800b27e0 con 0x7f87800ae060 2024-03-16T10:11:59.856 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T10:11:59.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f877dd9b640 1 --2- 172.21.15.17:0/3210410528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87800ae060 0x7f8780072b80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8770009e50 tx=0x7f877001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0caa36b8c36a63b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:59.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f877d59a640 1 -- 172.21.15.17:0/3210410528 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87700a58e0 con 0x7f87800ae060 2024-03-16T10:11:59.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f877d59a640 1 -- 172.21.15.17:0/3210410528 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87700a5a80 con 0x7f87800ae060 2024-03-16T10:11:59.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f877d59a640 1 -- 172.21.15.17:0/3210410528 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87700a5e50 con 0x7f87800ae060 2024-03-16T10:11:59.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 -- 172.21.15.17:0/3210410528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87800ae060 msgr2=0x7f8780072b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:59.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 --2- 172.21.15.17:0/3210410528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87800ae060 0x7f8780072b80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8770009e50 tx=0x7f877001c9f0 comp rx=0 tx=0).stop 2024-03-16T10:11:59.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 -- 172.21.15.17:0/3210410528 shutdown_connections 2024-03-16T10:11:59.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 --2- 172.21.15.17:0/3210410528 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87800ae060 0x7f8780072b80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:59.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 --2- 172.21.15.17:0/3210410528 >> v2:172.21.15.174:3300/0 conn(0x7f87800b2400 0x7f87801452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:59.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 -- 172.21.15.17:0/3210410528 >> 172.21.15.17:0/3210410528 conn(0x7f87800b8e90 msgr2=0x7f87800b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:11:59.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 -- 172.21.15.17:0/3210410528 shutdown_connections 2024-03-16T10:11:59.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 -- 172.21.15.17:0/3210410528 wait complete. 2024-03-16T10:11:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 Processor -- start 2024-03-16T10:11:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 -- start start 2024-03-16T10:11:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87800730c0 0x7f8780139a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87800ae060 0x7f87801e2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87800b2400 0x7f87801e26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8780145980 con 0x7f87800730c0 2024-03-16T10:11:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8780145800 con 0x7f87800b2400 2024-03-16T10:11:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.851+0000 7f8784a18640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8780152be0 con 0x7f87800ae060 2024-03-16T10:11:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f877e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87800730c0 0x7f8780139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f877e59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87800730c0 0x7f8780139a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55330/0 (socket says 172.21.15.17:55330) 2024-03-16T10:11:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f877e59c640 1 -- 172.21.15.17:0/163441120 learned_addr learned my addr 172.21.15.17:0/163441120 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:11:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f877dd9b640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87800ae060 0x7f87801e2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f877ed9d640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87800b2400 0x7f87801e26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:59.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f877e59c640 1 -- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87800ae060 msgr2=0x7f87801e2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f877e59c640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87800ae060 0x7f87801e2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f877e59c640 1 -- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87800b2400 msgr2=0x7f87801e26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:11:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f877e59c640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87800b2400 0x7f87801e26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:11:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f877e59c640 1 -- 172.21.15.17:0/163441120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8780140e60 con 0x7f87800730c0 2024-03-16T10:11:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f877dd9b640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87800ae060 0x7f87801e2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:11:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f877e59c640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87800730c0 0x7f8780139a10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f876800d620 tx=0x7f876800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f8776ffd640 1 -- 172.21.15.17:0/163441120 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876800b620 con 0x7f87800730c0 2024-03-16T10:11:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f8776ffd640 1 -- 172.21.15.17:0/163441120 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8768019070 con 0x7f87800730c0 2024-03-16T10:11:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f8784a18640 1 -- 172.21.15.17:0/163441120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878013d6b0 con 0x7f87800730c0 2024-03-16T10:11:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f8776ffd640 1 -- 172.21.15.17:0/163441120 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876800ba20 con 0x7f87800730c0 2024-03-16T10:11:59.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f8784a18640 1 -- 172.21.15.17:0/163441120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8780140b80 con 0x7f87800730c0 2024-03-16T10:11:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f8784a18640 1 -- 172.21.15.17:0/163441120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f878013ed30 con 0x7f87800730c0 2024-03-16T10:11:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f8776ffd640 1 -- 172.21.15.17:0/163441120 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f876800bbc0 con 0x7f87800730c0 2024-03-16T10:11:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f8776ffd640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8760067690 0x7f8760069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:11:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f8776ffd640 1 -- 172.21.15.17:0/163441120 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5795+0+0 (secure 0 0 0) 0x7f8768085cb0 con 0x7f87800730c0 2024-03-16T10:11:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.855+0000 7f877dd9b640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8760067690 0x7f8760069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:11:59.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.859+0000 7f877dd9b640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8760067690 0x7f8760069b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f87700ae000 tx=0x7f87700a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:11:59.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:11:59.859+0000 7f8776ffd640 1 -- 172.21.15.17:0/163441120 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8768056960 con 0x7f87800730c0 2024-03-16T10:12:00.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.019+0000 7f8784a18640 1 -- 172.21.15.17:0/163441120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8780068360 con 0x7f87800730c0 2024-03-16T10:12:00.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.643+0000 7f8776ffd640 1 -- 172.21.15.17:0/163441120 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v32) ==== 181+0+0 (secure 0 0 0) 0x7f8768056260 con 0x7f87800730c0 2024-03-16T10:12:00.651 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-03-16T10:12:00.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.647+0000 7f8784a18640 1 -- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8760067690 msgr2=0x7f8760069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:00.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.647+0000 7f8784a18640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8760067690 0x7f8760069b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f87700ae000 tx=0x7f87700a7070 comp rx=0 tx=0).stop 2024-03-16T10:12:00.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.647+0000 7f8784a18640 1 -- 172.21.15.17:0/163441120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87800730c0 msgr2=0x7f8780139a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:00.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.647+0000 7f8784a18640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87800730c0 0x7f8780139a10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f876800d620 tx=0x7f876800dae0 comp rx=0 tx=0).stop 2024-03-16T10:12:00.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.647+0000 7f8784a18640 1 -- 172.21.15.17:0/163441120 shutdown_connections 2024-03-16T10:12:00.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.647+0000 7f8784a18640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8760067690 0x7f8760069b50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:00.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.647+0000 7f8784a18640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87800b2400 0x7f87801e26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:00.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.647+0000 7f8784a18640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f87800ae060 0x7f87801e2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:00.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.647+0000 7f8784a18640 1 --2- 172.21.15.17:0/163441120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f87800730c0 0x7f8780139a10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:00.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.647+0000 7f8784a18640 1 -- 172.21.15.17:0/163441120 >> 172.21.15.17:0/163441120 conn(0x7f87800b8e90 msgr2=0x7f87800b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:00.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.647+0000 7f8784a18640 1 -- 172.21.15.17:0/163441120 shutdown_connections 2024-03-16T10:12:00.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.647+0000 7f8784a18640 1 -- 172.21.15.17:0/163441120 wait complete. 2024-03-16T10:12:00.681 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-16T10:12:00.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5ac8d4640 1 Processor -- start 2024-03-16T10:12:00.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5ac8d4640 1 -- start start 2024-03-16T10:12:00.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5ac8d4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a80b2680 0x7fa5a80b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:00.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5ac8d4640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a80b1ac0 con 0x7fa5a80b4800 2024-03-16T10:12:00.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5ac8d4640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a80b4be0 con 0x7fa5a8147ca0 2024-03-16T10:12:00.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5ac8d4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a8148840 con 0x7fa5a80b2680 2024-03-16T10:12:00.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a659c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa5a80b4800 0x7fa5a80b1f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54038/0 (socket says 172.21.15.17:54038) 2024-03-16T10:12:00.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a659c640 1 -- 172.21.15.17:0/520729278 learned_addr learned my addr 172.21.15.17:0/520729278 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:12:00.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 -- 172.21.15.17:0/520729278 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3168298957 0 0) 0x7fa5a80b1ac0 con 0x7fa5a80b4800 2024-03-16T10:12:00.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 -- 172.21.15.17:0/520729278 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa590003530 con 0x7fa5a80b4800 2024-03-16T10:12:00.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a5d9b640 1 --2- 172.21.15.17:0/520729278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a80b2680 0x7fa5a80b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:00.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 -- 172.21.15.17:0/520729278 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1035726328 0 0) 0x7fa590003530 con 0x7fa5a80b4800 2024-03-16T10:12:00.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 -- 172.21.15.17:0/520729278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a80b2680 msgr2=0x7fa5a80b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:00.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 --2- 172.21.15.17:0/520729278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a80b2680 0x7fa5a80b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:00.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 -- 172.21.15.17:0/520729278 >> v1:172.21.15.174:6789/0 conn(0x7fa5a8147ca0 legacy=0x7fa5a8148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:00.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 -- 172.21.15.17:0/520729278 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a80b1220 con 0x7fa5a80b4800 2024-03-16T10:12:00.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 -- 172.21.15.17:0/520729278 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fa598001680 con 0x7fa5a80b4800 2024-03-16T10:12:00.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 -- 172.21.15.17:0/520729278 >> v1:172.21.15.17:6789/0 conn(0x7fa5a80b4800 legacy=0x7fa5a80b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:00.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 --2- 172.21.15.17:0/520729278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5900039f0 0x7fa590003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:00.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 --2- 172.21.15.17:0/520729278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa590004810 0x7fa590004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:00.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 --2- 172.21.15.17:0/520729278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a80b2680 0x7fa59000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:00.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 -- 172.21.15.17:0/520729278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa590005500 con 0x7fa5900039f0 2024-03-16T10:12:00.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 -- 172.21.15.17:0/520729278 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa59000bf10 con 0x7fa590004810 2024-03-16T10:12:00.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a559a640 1 -- 172.21.15.17:0/520729278 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa59000c090 con 0x7fa5a80b2680 2024-03-16T10:12:00.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a659c640 1 --2- 172.21.15.17:0/520729278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5900039f0 0x7fa590003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:00.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a6d9d640 1 --2- 172.21.15.17:0/520729278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a80b2680 0x7fa59000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:00.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a5d9b640 1 --2- 172.21.15.17:0/520729278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa590004810 0x7fa590004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:00.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a659c640 1 -- 172.21.15.17:0/520729278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a80b2680 msgr2=0x7fa59000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:00.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a659c640 1 --2- 172.21.15.17:0/520729278 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5a80b2680 0x7fa59000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:00.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a659c640 1 -- 172.21.15.17:0/520729278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa590004810 msgr2=0x7fa590004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:00.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a659c640 1 --2- 172.21.15.17:0/520729278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa590004810 0x7fa590004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:00.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a659c640 1 -- 172.21.15.17:0/520729278 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa590003530 con 0x7fa5900039f0 2024-03-16T10:12:00.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.807+0000 7fa5a5d9b640 1 --2- 172.21.15.17:0/520729278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa590004810 0x7fa590004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:12:00.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5a659c640 1 --2- 172.21.15.17:0/520729278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5900039f0 0x7fa590003e10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa598007cc0 tx=0x7fa59801ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76cc4ac0e189d3d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:00.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5a559a640 1 -- 172.21.15.17:0/520729278 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa598007d00 con 0x7fa5900039f0 2024-03-16T10:12:00.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5a559a640 1 -- 172.21.15.17:0/520729278 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5980ab020 con 0x7fa5900039f0 2024-03-16T10:12:00.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5a559a640 1 -- 172.21.15.17:0/520729278 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5980a7b80 con 0x7fa5900039f0 2024-03-16T10:12:00.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/520729278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5900039f0 msgr2=0x7fa590003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:00.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5ac8d4640 1 --2- 172.21.15.17:0/520729278 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5900039f0 0x7fa590003e10 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa598007cc0 tx=0x7fa59801ee90 comp rx=0 tx=0).stop 2024-03-16T10:12:00.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5a6d9d640 1 -- 172.21.15.17:0/520729278 reap_dead start 2024-03-16T10:12:00.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/520729278 shutdown_connections 2024-03-16T10:12:00.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/520729278 >> 172.21.15.17:0/520729278 conn(0x7fa5a80b8e90 msgr2=0x7fa5a80b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:00.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/520729278 shutdown_connections 2024-03-16T10:12:00.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/520729278 wait complete. 2024-03-16T10:12:00.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5ac8d4640 1 Processor -- start 2024-03-16T10:12:00.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5ac8d4640 1 -- start start 2024-03-16T10:12:00.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5ac8d4640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5900039f0 0x7fa5a806f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:00.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5ac8d4640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a8071280 0x7fa5a8071700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:00.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5ac8d4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5a813b820 0x7fa5a813bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:00.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5ac8d4640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a80b3e20 con 0x7fa5a8071280 2024-03-16T10:12:00.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5ac8d4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a80b3ca0 con 0x7fa5a813b820 2024-03-16T10:12:00.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5ac8d4640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a80b3fa0 con 0x7fa5900039f0 2024-03-16T10:12:00.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5a5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a8071280 0x7fa5a8071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:00.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5a5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a8071280 0x7fa5a8071700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55348/0 (socket says 172.21.15.17:55348) 2024-03-16T10:12:00.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.811+0000 7fa5a5d9b640 1 -- 172.21.15.17:0/3367079313 learned_addr learned my addr 172.21.15.17:0/3367079313 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:00.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa5a659c640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5900039f0 0x7fa5a806f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:00.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa5a6d9d640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5a813b820 0x7fa5a813bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:00.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa5a5d9b640 1 -- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5900039f0 msgr2=0x7fa5a806f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:00.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa5a5d9b640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5900039f0 0x7fa5a806f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:00.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa5a5d9b640 1 -- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5a813b820 msgr2=0x7fa5a813bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:00.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa5a5d9b640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5a813b820 0x7fa5a813bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:00.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa5a5d9b640 1 -- 172.21.15.17:0/3367079313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a8070df0 con 0x7fa5a8071280 2024-03-16T10:12:00.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa5a659c640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5900039f0 0x7fa5a806f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:12:00.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa5a5d9b640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a8071280 0x7fa5a8071700 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00e870 tx=0x7fa59c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:00.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa596ffd640 1 -- 172.21.15.17:0/3367079313 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c017070 con 0x7fa5a8071280 2024-03-16T10:12:00.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa596ffd640 1 -- 172.21.15.17:0/3367079313 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59c00c8d0 con 0x7fa5a8071280 2024-03-16T10:12:00.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa596ffd640 1 -- 172.21.15.17:0/3367079313 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c00f040 con 0x7fa5a8071280 2024-03-16T10:12:00.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/3367079313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a813c1e0 con 0x7fa5a8071280 2024-03-16T10:12:00.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/3367079313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a81dc250 con 0x7fa5a8071280 2024-03-16T10:12:00.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/3367079313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a8068360 con 0x7fa5a8071280 2024-03-16T10:12:00.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa596ffd640 1 -- 172.21.15.17:0/3367079313 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa59c01a070 con 0x7fa5a8071280 2024-03-16T10:12:00.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa596ffd640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa588067690 0x7fa588069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:00.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa596ffd640 1 -- 172.21.15.17:0/3367079313 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5357+0+0 (secure 0 0 0) 0x7fa59c085260 con 0x7fa5a8071280 2024-03-16T10:12:00.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa5a659c640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa588067690 0x7fa588069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:00.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.815+0000 7fa5a659c640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa588067690 0x7fa588069b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa5980ac000 tx=0x7fa5a80aedf0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:00.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.819+0000 7fa596ffd640 1 -- 172.21.15.17:0/3367079313 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa5a8070df0 con 0x7fa5a8071280 2024-03-16T10:12:00.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:00.991+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/3367079313 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa5a81dcd30 con 0x7fa5a8071280 2024-03-16T10:12:01.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.647+0000 7fa596ffd640 1 -- 172.21.15.17:0/3367079313 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v33) ==== 169+0+0 (secure 0 0 0) 0x7fa59c00ee60 con 0x7fa5a8071280 2024-03-16T10:12:01.654 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-03-16T10:12:01.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.651+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa588067690 msgr2=0x7fa588069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:01.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.651+0000 7fa5ac8d4640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa588067690 0x7fa588069b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa5980ac000 tx=0x7fa5a80aedf0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.651+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/3367079313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a8071280 msgr2=0x7fa5a8071700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:01.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.651+0000 7fa5ac8d4640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a8071280 0x7fa5a8071700 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa59c00e870 tx=0x7fa59c00ed30 comp rx=0 tx=0).stop 2024-03-16T10:12:01.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.651+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/3367079313 shutdown_connections 2024-03-16T10:12:01.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.651+0000 7fa5ac8d4640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa588067690 0x7fa588069b50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.651+0000 7fa5ac8d4640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5a813b820 0x7fa5a813bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.651+0000 7fa5ac8d4640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa5a8071280 0x7fa5a8071700 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.651+0000 7fa5ac8d4640 1 --2- 172.21.15.17:0/3367079313 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa5900039f0 0x7fa5a806f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.651+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/3367079313 >> 172.21.15.17:0/3367079313 conn(0x7fa5a80b8e90 msgr2=0x7fa5a80b5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:01.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.651+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/3367079313 shutdown_connections 2024-03-16T10:12:01.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.651+0000 7fa5ac8d4640 1 -- 172.21.15.17:0/3367079313 wait complete. 2024-03-16T10:12:01.681 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-16T10:12:01.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90b590d640 1 Processor -- start 2024-03-16T10:12:01.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90b590d640 1 -- start start 2024-03-16T10:12:01.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90b590d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90b00b4800 0x7f90b00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:01.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90b590d640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f90b00b1f70 0x7f90b00b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:01.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90b590d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90b0147ca0 0x7f90b0148130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:01.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90b590d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f90b00b1ac0 con 0x7f90b0147ca0 2024-03-16T10:12:01.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90b590d640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f90b00b4be0 con 0x7f90b00b1f70 2024-03-16T10:12:01.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90b590d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f90b00b28f0 con 0x7f90b00b4800 2024-03-16T10:12:01.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90af7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90b0147ca0 0x7f90b0148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90af7fe640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f90b0147ca0 0x7f90b0148130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55360/0 (socket says 172.21.15.17:55360) 2024-03-16T10:12:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90af7fe640 1 -- 172.21.15.17:0/394290109 learned_addr learned my addr 172.21.15.17:0/394290109 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90ae7fc640 1 --2- 172.21.15.17:0/394290109 >> v2:172.21.15.174:3300/0 conn(0x7f90b00b1f70 0x7f90b00b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90aeffd640 1 --2- 172.21.15.17:0/394290109 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90b00b4800 0x7f90b00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90af7fe640 1 -- 172.21.15.17:0/394290109 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90b00b4800 msgr2=0x7f90b00b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90af7fe640 1 --2- 172.21.15.17:0/394290109 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90b00b4800 0x7f90b00b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90af7fe640 1 -- 172.21.15.17:0/394290109 >> v2:172.21.15.174:3300/0 conn(0x7f90b00b1f70 msgr2=0x7f90b00b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90af7fe640 1 --2- 172.21.15.17:0/394290109 >> v2:172.21.15.174:3300/0 conn(0x7f90b00b1f70 0x7f90b00b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.803+0000 7f90af7fe640 1 -- 172.21.15.17:0/394290109 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b00b1220 con 0x7f90b0147ca0 2024-03-16T10:12:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90af7fe640 1 --2- 172.21.15.17:0/394290109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b0147ca0 0x7f90b0148130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f90b006dfc0 tx=0x7f90a001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0c6961e985f994 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90adffb640 1 -- 172.21.15.17:0/394290109 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a00b2070 con 0x7f90b0147ca0 2024-03-16T10:12:01.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90adffb640 1 -- 172.21.15.17:0/394290109 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a00a9070 con 0x7f90b0147ca0 2024-03-16T10:12:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90adffb640 1 -- 172.21.15.17:0/394290109 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a00ad880 con 0x7f90b0147ca0 2024-03-16T10:12:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90b590d640 1 -- 172.21.15.17:0/394290109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b0147ca0 msgr2=0x7f90b0148130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:01.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90b590d640 1 --2- 172.21.15.17:0/394290109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b0147ca0 0x7f90b0148130 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f90b006dfc0 tx=0x7f90a001c490 comp rx=0 tx=0).stop 2024-03-16T10:12:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90b590d640 1 -- 172.21.15.17:0/394290109 shutdown_connections 2024-03-16T10:12:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90b590d640 1 --2- 172.21.15.17:0/394290109 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b0147ca0 0x7f90b0148130 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90b590d640 1 --2- 172.21.15.17:0/394290109 >> v2:172.21.15.174:3300/0 conn(0x7f90b00b1f70 0x7f90b00b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90b590d640 1 --2- 172.21.15.17:0/394290109 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90b00b4800 0x7f90b00b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90b590d640 1 -- 172.21.15.17:0/394290109 >> 172.21.15.17:0/394290109 conn(0x7f90b00b8e90 msgr2=0x7f90b00b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90b590d640 1 -- 172.21.15.17:0/394290109 shutdown_connections 2024-03-16T10:12:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90b590d640 1 -- 172.21.15.17:0/394290109 wait complete. 2024-03-16T10:12:01.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90b590d640 1 Processor -- start 2024-03-16T10:12:01.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.807+0000 7f90b590d640 1 -- start start 2024-03-16T10:12:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90b590d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b00b1f70 0x7f90b0145110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:01.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90b590d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b00b4800 0x7f90b0144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90b590d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90b0147ca0 0x7f90b013bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90b590d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01487f0 con 0x7f90b00b1f70 2024-03-16T10:12:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90b590d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90b0148670 con 0x7f90b00b4800 2024-03-16T10:12:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90b590d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f90b006cee0 con 0x7f90b0147ca0 2024-03-16T10:12:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90aeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b00b1f70 0x7f90b0145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90aeffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b00b1f70 0x7f90b0145110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55366/0 (socket says 172.21.15.17:55366) 2024-03-16T10:12:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90aeffd640 1 -- 172.21.15.17:0/2826019646 learned_addr learned my addr 172.21.15.17:0/2826019646 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90ae7fc640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b00b4800 0x7f90b0144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90af7fe640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90b0147ca0 0x7f90b013bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90aeffd640 1 -- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90b0147ca0 msgr2=0x7f90b013bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90aeffd640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90b0147ca0 0x7f90b013bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90aeffd640 1 -- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b00b4800 msgr2=0x7f90b0144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:01.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90aeffd640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b00b4800 0x7f90b0144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90aeffd640 1 -- 172.21.15.17:0/2826019646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b0143250 con 0x7f90b00b1f70 2024-03-16T10:12:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90ae7fc640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b00b4800 0x7f90b0144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:12:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90af7fe640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90b0147ca0 0x7f90b013bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:12:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90aeffd640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b00b1f70 0x7f90b0145110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f90a8007ef0 tx=0x7f90a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90a7fff640 1 -- 172.21.15.17:0/2826019646 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8014620 con 0x7f90b00b1f70 2024-03-16T10:12:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90a7fff640 1 -- 172.21.15.17:0/2826019646 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a8019070 con 0x7f90b00b1f70 2024-03-16T10:12:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90a7fff640 1 -- 172.21.15.17:0/2826019646 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8014a00 con 0x7f90b00b1f70 2024-03-16T10:12:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90b590d640 1 -- 172.21.15.17:0/2826019646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b01459a0 con 0x7f90b00b1f70 2024-03-16T10:12:01.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90b590d640 1 -- 172.21.15.17:0/2826019646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b0145650 con 0x7f90b00b1f70 2024-03-16T10:12:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90b590d640 1 -- 172.21.15.17:0/2826019646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b0068360 con 0x7f90b00b1f70 2024-03-16T10:12:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90a7fff640 1 -- 172.21.15.17:0/2826019646 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f90a8014ba0 con 0x7f90b00b1f70 2024-03-16T10:12:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90a7fff640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9090067690 0x7f9090069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.811+0000 7f90a7fff640 1 -- 172.21.15.17:0/2826019646 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f90a8086b70 con 0x7f90b00b1f70 2024-03-16T10:12:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.815+0000 7f90ae7fc640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9090067690 0x7f9090069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:01.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.815+0000 7f90ae7fc640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9090067690 0x7f9090069b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f90b0143db0 tx=0x7f9098008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:01.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.815+0000 7f90a7fff640 1 -- 172.21.15.17:0/2826019646 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f90b01459a0 con 0x7f90b00b1f70 2024-03-16T10:12:01.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.979+0000 7f90b590d640 1 -- 172.21.15.17:0/2826019646 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f90b00b6c70 con 0x7f90b00b1f70 2024-03-16T10:12:01.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.983+0000 7f90a7fff640 1 -- 172.21.15.17:0/2826019646 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f90a8057c60 con 0x7f90b00b1f70 2024-03-16T10:12:01.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.983+0000 7f90b590d640 1 -- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9090067690 msgr2=0x7f9090069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:01.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.983+0000 7f90b590d640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9090067690 0x7f9090069b50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f90b0143db0 tx=0x7f9098008040 comp rx=0 tx=0).stop 2024-03-16T10:12:01.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.987+0000 7f90b590d640 1 -- 172.21.15.17:0/2826019646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b00b1f70 msgr2=0x7f90b0145110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:01.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.987+0000 7f90b590d640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b00b1f70 0x7f90b0145110 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f90a8007ef0 tx=0x7f90a800e4b0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.987+0000 7f90b590d640 1 -- 172.21.15.17:0/2826019646 shutdown_connections 2024-03-16T10:12:01.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.987+0000 7f90b590d640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9090067690 0x7f9090069b50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.987+0000 7f90b590d640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f90b0147ca0 0x7f90b013bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.987+0000 7f90b590d640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b00b4800 0x7f90b0144250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.987+0000 7f90b590d640 1 --2- 172.21.15.17:0/2826019646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90b00b1f70 0x7f90b0145110 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:01.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.987+0000 7f90b590d640 1 -- 172.21.15.17:0/2826019646 >> 172.21.15.17:0/2826019646 conn(0x7f90b00b8e90 msgr2=0x7f90b0071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:01.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.987+0000 7f90b590d640 1 -- 172.21.15.17:0/2826019646 shutdown_connections 2024-03-16T10:12:01.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:01.987+0000 7f90b590d640 1 -- 172.21.15.17:0/2826019646 wait complete. 2024-03-16T10:12:02.009 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2024-03-16T10:12:02.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.127+0000 7f812d3c1640 1 Processor -- start 2024-03-16T10:12:02.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.127+0000 7f812d3c1640 1 -- start start 2024-03-16T10:12:02.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.127+0000 7f812d3c1640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f81280b1950 0x7f81280b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:02.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.127+0000 7f812d3c1640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81280b1d30 con 0x7f81280b2080 2024-03-16T10:12:02.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.127+0000 7f812d3c1640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8128143940 con 0x7f8128071bb0 2024-03-16T10:12:02.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.127+0000 7f812d3c1640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f81280b2b90 con 0x7f81280b1950 2024-03-16T10:12:02.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.127+0000 7f8126ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f81280b1950 0x7f81280b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:02.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.127+0000 7f8126ffd640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f81280b1950 0x7f81280b2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55368/0 (socket says 172.21.15.17:55368) 2024-03-16T10:12:02.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.127+0000 7f8126ffd640 1 -- 172.21.15.17:0/2970705170 learned_addr learned my addr 172.21.15.17:0/2970705170 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:02.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.127+0000 7f81267fc640 1 --1- 172.21.15.17:0/2970705170 >> v1:172.21.15.17:6789/0 conn(0x7f81280b2080 0x7f81280affc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54042/0 (socket says 172.21.15.17:54042) 2024-03-16T10:12:02.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.127+0000 7f8126ffd640 1 -- 172.21.15.17:0/2970705170 >> v1:172.21.15.174:6789/0 conn(0x7f8128071bb0 legacy=0x7f812814e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:02.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.127+0000 7f8126ffd640 1 -- 172.21.15.17:0/2970705170 >> v1:172.21.15.17:6789/0 conn(0x7f81280b2080 legacy=0x7f81280affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:02.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.127+0000 7f8126ffd640 1 -- 172.21.15.17:0/2970705170 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8128143c00 con 0x7f81280b1950 2024-03-16T10:12:02.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.127+0000 7f8125ffb640 1 -- 172.21.15.17:0/2970705170 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2728476414 0 0) 0x7f81280b1d30 con 0x7f81280b2080 2024-03-16T10:12:02.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.131+0000 7f8126ffd640 1 --2- 172.21.15.17:0/2970705170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81280b1950 0x7f81280b2530 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8118009e50 tx=0x7f811801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da1e6e669f5122d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:02.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.131+0000 7f8125ffb640 1 -- 172.21.15.17:0/2970705170 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81180a57a0 con 0x7f81280b1950 2024-03-16T10:12:02.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.131+0000 7f8125ffb640 1 -- 172.21.15.17:0/2970705170 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81180a5940 con 0x7f81280b1950 2024-03-16T10:12:02.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.131+0000 7f8125ffb640 1 -- 172.21.15.17:0/2970705170 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81180a5d10 con 0x7f81280b1950 2024-03-16T10:12:02.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.131+0000 7f812d3c1640 1 -- 172.21.15.17:0/2970705170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81280b1950 msgr2=0x7f81280b2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:02.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.131+0000 7f812d3c1640 1 --2- 172.21.15.17:0/2970705170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81280b1950 0x7f81280b2530 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8118009e50 tx=0x7f811801c8e0 comp rx=0 tx=0).stop 2024-03-16T10:12:02.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.131+0000 7f812d3c1640 1 -- 172.21.15.17:0/2970705170 shutdown_connections 2024-03-16T10:12:02.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.131+0000 7f812d3c1640 1 --2- 172.21.15.17:0/2970705170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81280b1950 0x7f81280b2530 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:02.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.131+0000 7f812d3c1640 1 -- 172.21.15.17:0/2970705170 >> 172.21.15.17:0/2970705170 conn(0x7f81280b55c0 msgr2=0x7f81280b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:02.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.131+0000 7f812d3c1640 1 -- 172.21.15.17:0/2970705170 shutdown_connections 2024-03-16T10:12:02.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.131+0000 7f812d3c1640 1 -- 172.21.15.17:0/2970705170 wait complete. 2024-03-16T10:12:02.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.131+0000 7f812d3c1640 1 Processor -- start 2024-03-16T10:12:02.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.131+0000 7f812d3c1640 1 -- start start 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f812d3c1640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8128071bb0 0x7f81281e1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f812d3c1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81280b1950 0x7f81280aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f812d3c1640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f81280b2080 0x7f812813fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f812d3c1640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8128072550 con 0x7f8128071bb0 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f812d3c1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81280723d0 con 0x7f81280b1950 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f812d3c1640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f812806d4f0 con 0x7f81280b2080 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f8126ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8128071bb0 0x7f81281e1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f8126ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8128071bb0 0x7f81281e1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55380/0 (socket says 172.21.15.17:55380) 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f8126ffd640 1 -- 172.21.15.17:0/170635578 learned_addr learned my addr 172.21.15.17:0/170635578 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f81277fe640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f81280b2080 0x7f812813fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f81267fc640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81280b1950 0x7f81280aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f8126ffd640 1 -- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f81280b2080 msgr2=0x7f812813fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f8126ffd640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f81280b2080 0x7f812813fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f8126ffd640 1 -- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81280b1950 msgr2=0x7f81280aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:02.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f8126ffd640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81280b1950 0x7f81280aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:02.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f8126ffd640 1 -- 172.21.15.17:0/170635578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81280aac10 con 0x7f8128071bb0 2024-03-16T10:12:02.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f81267fc640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81280b1950 0x7f81280aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:12:02.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f8126ffd640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8128071bb0 0x7f81281e1820 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f81180af040 tx=0x7f8118019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:02.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f810f7fe640 1 -- 172.21.15.17:0/170635578 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81180bc020 con 0x7f8128071bb0 2024-03-16T10:12:02.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f810f7fe640 1 -- 172.21.15.17:0/170635578 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81180b8670 con 0x7f8128071bb0 2024-03-16T10:12:02.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f810f7fe640 1 -- 172.21.15.17:0/170635578 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81180b8a60 con 0x7f8128071bb0 2024-03-16T10:12:02.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f812d3c1640 1 -- 172.21.15.17:0/170635578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81280aaec0 con 0x7f8128071bb0 2024-03-16T10:12:02.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f812d3c1640 1 -- 172.21.15.17:0/170635578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f812813cba0 con 0x7f8128071bb0 2024-03-16T10:12:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f812d3c1640 1 -- 172.21.15.17:0/170635578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8128068360 con 0x7f8128071bb0 2024-03-16T10:12:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f810f7fe640 1 -- 172.21.15.17:0/170635578 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f81180bf020 con 0x7f8128071bb0 2024-03-16T10:12:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f810f7fe640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8114067660 0x7f8114069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f810f7fe640 1 -- 172.21.15.17:0/170635578 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f8118120790 con 0x7f8128071bb0 2024-03-16T10:12:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.135+0000 7f81267fc640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8114067660 0x7f8114069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:02.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.139+0000 7f81267fc640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8114067660 0x7f8114069b20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f81280b3d80 tx=0x7f811c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:02.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.139+0000 7f810f7fe640 1 -- 172.21.15.17:0/170635578 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f81180c6030 con 0x7f8128071bb0 2024-03-16T10:12:02.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.379+0000 7f812d3c1640 1 -- 172.21.15.17:0/170635578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) -- 0x7f8128139680 con 0x7f8128071bb0 2024-03-16T10:12:02.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.651+0000 7f810f7fe640 1 -- 172.21.15.17:0/170635578 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v84) ==== 134+0+0 (secure 0 0 0) 0x7f8118016620 con 0x7f8128071bb0 2024-03-16T10:12:02.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.655+0000 7f812d3c1640 1 -- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8114067660 msgr2=0x7f8114069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:02.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.655+0000 7f812d3c1640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8114067660 0x7f8114069b20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f81280b3d80 tx=0x7f811c009040 comp rx=0 tx=0).stop 2024-03-16T10:12:02.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.655+0000 7f812d3c1640 1 -- 172.21.15.17:0/170635578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8128071bb0 msgr2=0x7f81281e1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:02.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.655+0000 7f812d3c1640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8128071bb0 0x7f81281e1820 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f81180af040 tx=0x7f8118019680 comp rx=0 tx=0).stop 2024-03-16T10:12:02.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.655+0000 7f812d3c1640 1 -- 172.21.15.17:0/170635578 shutdown_connections 2024-03-16T10:12:02.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.655+0000 7f812d3c1640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8114067660 0x7f8114069b20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:02.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.655+0000 7f812d3c1640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f81280b2080 0x7f812813fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:02.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.655+0000 7f812d3c1640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81280b1950 0x7f81280aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:02.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.655+0000 7f812d3c1640 1 --2- 172.21.15.17:0/170635578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8128071bb0 0x7f81281e1820 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:02.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.655+0000 7f812d3c1640 1 -- 172.21.15.17:0/170635578 >> 172.21.15.17:0/170635578 conn(0x7f81280b55c0 msgr2=0x7f81280b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:02.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.655+0000 7f812d3c1640 1 -- 172.21.15.17:0/170635578 shutdown_connections 2024-03-16T10:12:02.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:02.655+0000 7f812d3c1640 1 -- 172.21.15.17:0/170635578 wait complete. 2024-03-16T10:12:02.683 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed ... ERROR 2024-03-16T10:12:02.683 INFO:tasks.cephfs_test_runner: 2024-03-16T10:12:02.683 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner: r.wait() 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-03-16T10:12:02.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi017 with status 127: 'cephfs-shell -b /tmp/test-cephfs8olo1904' 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner: 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner:Ran 1 test in 43.714s 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner: 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner: 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2024-03-16T10:12:02.685 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2024-03-16T10:12:02.686 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2024-03-16T10:12:02.686 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2024-03-16T10:12:02.686 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-16T10:12:02.686 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-16T10:12:02.686 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run 2024-03-16T10:12:02.686 INFO:tasks.cephfs_test_runner: r.wait() 2024-03-16T10:12:02.686 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait 2024-03-16T10:12:02.686 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-03-16T10:12:02.686 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-03-16T10:12:02.686 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-03-16T10:12:02.686 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi017 with status 127: 'cephfs-shell -b /tmp/test-cephfs8olo1904' 2024-03-16T10:12:02.686 INFO:tasks.cephfs_test_runner: 2024-03-16T10:12:02.686 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-16T10:12:02.916 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=f2e18ce7b61f40ef91ba7c653e2538a2 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-16T10:12:02.918 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-16T10:12:02.926 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-16T10:12:02.933 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-16T10:12:02.933 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-03-16T10:12:02.933 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T10:12:02.947 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339612k,nr_inodes=4084903,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15779 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275312k,nr_inodes=818828,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-16T10:12:02.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:12:02.947 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-03-16T10:12:02.993 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-16T10:12:02.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:12:02.994 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T10:12:03.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T10:12:03.050 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link delete ceph-brx 2024-03-16T10:12:03.266 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-16T10:12:03.274 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-16T10:12:03.274 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:12:03.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d327bc640 1 Processor -- start 2024-03-16T10:12:03.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d327bc640 1 -- start start 2024-03-16T10:12:03.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d327bc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d2c072a80 0x7f2d2c072ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:03.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d327bc640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2c0b6350 con 0x7f2d2c06f670 2024-03-16T10:12:03.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d327bc640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2c0b6810 con 0x7f2d2c0b5f70 2024-03-16T10:12:03.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d327bc640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2c0b65c0 con 0x7f2d2c072a80 2024-03-16T10:12:03.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d30d59640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f2d2c06f670 0x7f2d2c06fad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54054/0 (socket says 172.21.15.17:54054) 2024-03-16T10:12:03.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d30d59640 1 -- 172.21.15.17:0/3788719846 learned_addr learned my addr 172.21.15.17:0/3788719846 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:12:03.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d237fe640 1 --2- 172.21.15.17:0/3788719846 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d2c072a80 0x7f2d2c072ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:03.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 -- 172.21.15.17:0/3788719846 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1430789290 0 0) 0x7f2d2c0b6350 con 0x7f2d2c06f670 2024-03-16T10:12:03.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 -- 172.21.15.17:0/3788719846 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d18003530 con 0x7f2d2c06f670 2024-03-16T10:12:03.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 -- 172.21.15.17:0/3788719846 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3865896113 0 0) 0x7f2d18003530 con 0x7f2d2c06f670 2024-03-16T10:12:03.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 -- 172.21.15.17:0/3788719846 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d2c072a80 msgr2=0x7f2d2c072ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:03.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 --2- 172.21.15.17:0/3788719846 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d2c072a80 0x7f2d2c072ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 -- 172.21.15.17:0/3788719846 >> v1:172.21.15.174:6789/0 conn(0x7f2d2c0b5f70 legacy=0x7f2d2c144660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 -- 172.21.15.17:0/3788719846 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c0b6af0 con 0x7f2d2c06f670 2024-03-16T10:12:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 -- 172.21.15.17:0/3788719846 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f2d24001a00 con 0x7f2d2c06f670 2024-03-16T10:12:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 -- 172.21.15.17:0/3788719846 >> v1:172.21.15.17:6789/0 conn(0x7f2d2c06f670 legacy=0x7f2d2c06fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 --2- 172.21.15.17:0/3788719846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d180039f0 0x7f2d18003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 --2- 172.21.15.17:0/3788719846 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d2c072a80 0x7f2d18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 --2- 172.21.15.17:0/3788719846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d18004d50 0x7f2d180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 -- 172.21.15.17:0/3788719846 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c0b65c0 con 0x7f2d180039f0 2024-03-16T10:12:03.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 -- 172.21.15.17:0/3788719846 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1800bb90 con 0x7f2d18004d50 2024-03-16T10:12:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2b7fe640 1 -- 172.21.15.17:0/3788719846 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1800bd90 con 0x7f2d2c072a80 2024-03-16T10:12:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2bfff640 1 --2- 172.21.15.17:0/3788719846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d180039f0 0x7f2d18003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d237fe640 1 --2- 172.21.15.17:0/3788719846 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d2c072a80 0x7f2d18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d30d59640 1 --2- 172.21.15.17:0/3788719846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d18004d50 0x7f2d180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2bfff640 1 -- 172.21.15.17:0/3788719846 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d2c072a80 msgr2=0x7f2d18004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2bfff640 1 --2- 172.21.15.17:0/3788719846 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d2c072a80 0x7f2d18004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2bfff640 1 -- 172.21.15.17:0/3788719846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d18004d50 msgr2=0x7f2d180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:03.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2bfff640 1 --2- 172.21.15.17:0/3788719846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d18004d50 0x7f2d180051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.403+0000 7f2d2bfff640 1 -- 172.21.15.17:0/3788719846 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d18003530 con 0x7f2d180039f0 2024-03-16T10:12:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d2bfff640 1 --2- 172.21.15.17:0/3788719846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d180039f0 0x7f2d18003e10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2d14001780 tx=0x7f2d1401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b4790ca81e7a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:03.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d2b7fe640 1 -- 172.21.15.17:0/3788719846 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d140013a0 con 0x7f2d180039f0 2024-03-16T10:12:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d2b7fe640 1 -- 172.21.15.17:0/3788719846 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d140052f0 con 0x7f2d180039f0 2024-03-16T10:12:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d2b7fe640 1 -- 172.21.15.17:0/3788719846 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14003840 con 0x7f2d180039f0 2024-03-16T10:12:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d327bc640 1 -- 172.21.15.17:0/3788719846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d180039f0 msgr2=0x7f2d18003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d327bc640 1 --2- 172.21.15.17:0/3788719846 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d180039f0 0x7f2d18003e10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2d14001780 tx=0x7f2d1401c420 comp rx=0 tx=0).stop 2024-03-16T10:12:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d30d59640 1 -- 172.21.15.17:0/3788719846 reap_dead start 2024-03-16T10:12:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d327bc640 1 -- 172.21.15.17:0/3788719846 shutdown_connections 2024-03-16T10:12:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d327bc640 1 -- 172.21.15.17:0/3788719846 >> 172.21.15.17:0/3788719846 conn(0x7f2d2c0b79d0 msgr2=0x7f2d2c0b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d327bc640 1 -- 172.21.15.17:0/3788719846 shutdown_connections 2024-03-16T10:12:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d327bc640 1 -- 172.21.15.17:0/3788719846 wait complete. 2024-03-16T10:12:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d327bc640 1 Processor -- start 2024-03-16T10:12:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d327bc640 1 -- start start 2024-03-16T10:12:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d327bc640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d180039f0 0x7f2d2c1e3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d327bc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d2c0b0800 0x7f2d2c0b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d327bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c0afef0 0x7f2d2c1472b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d327bc640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c153880 con 0x7f2d2c0afef0 2024-03-16T10:12:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d327bc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c153700 con 0x7f2d2c0b0800 2024-03-16T10:12:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d327bc640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2c153a00 con 0x7f2d180039f0 2024-03-16T10:12:03.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d30d59640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c0afef0 0x7f2d2c1472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:03.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d30d59640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c0afef0 0x7f2d2c1472b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55392/0 (socket says 172.21.15.17:55392) 2024-03-16T10:12:03.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d30d59640 1 -- 172.21.15.17:0/1453268131 learned_addr learned my addr 172.21.15.17:0/1453268131 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:03.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d237fe640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d2c0b0800 0x7f2d2c0b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:03.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.407+0000 7f2d2bfff640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d180039f0 0x7f2d2c1e3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:03.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d30d59640 1 -- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d180039f0 msgr2=0x7f2d2c1e3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:03.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d30d59640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d180039f0 0x7f2d2c1e3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d30d59640 1 -- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d2c0b0800 msgr2=0x7f2d2c0b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:03.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d30d59640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d2c0b0800 0x7f2d2c0b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d30d59640 1 -- 172.21.15.17:0/1453268131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c0b0ef0 con 0x7f2d2c0afef0 2024-03-16T10:12:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d237fe640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d2c0b0800 0x7f2d2c0b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:12:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d30d59640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c0afef0 0x7f2d2c1472b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2d24001590 tx=0x7f2d2400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d297fa640 1 -- 172.21.15.17:0/1453268131 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24004d70 con 0x7f2d2c0afef0 2024-03-16T10:12:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d297fa640 1 -- 172.21.15.17:0/1453268131 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d2400bb20 con 0x7f2d2c0afef0 2024-03-16T10:12:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d297fa640 1 -- 172.21.15.17:0/1453268131 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24007110 con 0x7f2d2c0afef0 2024-03-16T10:12:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d327bc640 1 -- 172.21.15.17:0/1453268131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2c0b11d0 con 0x7f2d2c0afef0 2024-03-16T10:12:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d327bc640 1 -- 172.21.15.17:0/1453268131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d2c06d7a0 con 0x7f2d2c0afef0 2024-03-16T10:12:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d327bc640 1 -- 172.21.15.17:0/1453268131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2c13fd80 con 0x7f2d2c0afef0 2024-03-16T10:12:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d297fa640 1 -- 172.21.15.17:0/1453268131 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f2d2401b060 con 0x7f2d2c0afef0 2024-03-16T10:12:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d297fa640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2d0c067660 0x7f2d0c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d297fa640 1 -- 172.21.15.17:0/1453268131 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f2d240854f0 con 0x7f2d2c0afef0 2024-03-16T10:12:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d2bfff640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2d0c067660 0x7f2d0c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:03.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.411+0000 7f2d2bfff640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2d0c067660 0x7f2d0c069b20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2d14001750 tx=0x7f2d140011a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:03.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.415+0000 7f2d297fa640 1 -- 172.21.15.17:0/1453268131 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2d24056500 con 0x7f2d2c0afef0 2024-03-16T10:12:03.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.571+0000 7f2d327bc640 1 -- 172.21.15.17:0/1453268131 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d2c06a130 con 0x7f2d0c067660 2024-03-16T10:12:03.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.575+0000 7f2d297fa640 1 -- 172.21.15.17:0/1453268131 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7f2d2c06a130 con 0x7f2d0c067660 2024-03-16T10:12:03.582 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:12:03.582 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-16T10:12:03.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.575+0000 7f2d327bc640 1 -- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2d0c067660 msgr2=0x7f2d0c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:03.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.575+0000 7f2d327bc640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2d0c067660 0x7f2d0c069b20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2d14001750 tx=0x7f2d140011a0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.575+0000 7f2d327bc640 1 -- 172.21.15.17:0/1453268131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c0afef0 msgr2=0x7f2d2c1472b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:03.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.575+0000 7f2d327bc640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c0afef0 0x7f2d2c1472b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2d24001590 tx=0x7f2d2400b4a0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.579+0000 7f2d327bc640 1 -- 172.21.15.17:0/1453268131 shutdown_connections 2024-03-16T10:12:03.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.579+0000 7f2d327bc640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2d0c067660 0x7f2d0c069b20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.579+0000 7f2d327bc640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2d2c0afef0 0x7f2d2c1472b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.579+0000 7f2d327bc640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d2c0b0800 0x7f2d2c0b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.579+0000 7f2d327bc640 1 --2- 172.21.15.17:0/1453268131 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2d180039f0 0x7f2d2c1e3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.579+0000 7f2d327bc640 1 -- 172.21.15.17:0/1453268131 >> 172.21.15.17:0/1453268131 conn(0x7f2d2c0b79d0 msgr2=0x7f2d2c0b5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:03.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.579+0000 7f2d327bc640 1 -- 172.21.15.17:0/1453268131 shutdown_connections 2024-03-16T10:12:03.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.579+0000 7f2d327bc640 1 -- 172.21.15.17:0/1453268131 wait complete. 2024-03-16T10:12:03.601 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-03-16T10:12:02.036551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236768,"kb_used_data":3968,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737952,"statfs":{"total":773094113280,"available":772851662848,"internally_reserved":0,"allocated":4063232,"data_stored":2289004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002167"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":189,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-16T10:11:58.876299+0000","last_change":"2024-03-16T10:10:48.248686+0000","last_active":"2024-03-16T10:11:58.876299+0000","last_peered":"2024-03-16T10:11:58.876299+0000","last_clean":"2024-03-16T10:11:58.876299+0000","last_became_active":"2024-03-16T10:10:48.243054+0000","last_became_peered":"2024-03-16T10:10:48.243054+0000","last_unstale":"2024-03-16T10:11:58.876299+0000","last_undegraded":"2024-03-16T10:11:58.876299+0000","last_fullsized":"2024-03-16T10:11:58.876299+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_clean_scrub_stamp":"2024-03-16T10:10:46.517938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:42.906473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":946176,"data_stored":725222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":135380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":135436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":134868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":376832,"data_stored":149066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":134834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":376832,"data_stored":149066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":942080,"data_stored":725132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:12:03.601 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:12:03.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedbc0be640 1 Processor -- start 2024-03-16T10:12:03.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedbc0be640 1 -- start start 2024-03-16T10:12:03.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedbc0be640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fedb406d750 0x7fedb406f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:03.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedbc0be640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fedb4145280 0x7fedb413f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:03.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedbc0be640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedb406db30 con 0x7fedb40b2820 2024-03-16T10:12:03.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedbc0be640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fedb40b2d30 con 0x7fedb4145280 2024-03-16T10:12:03.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedbc0be640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fedb406fcf0 con 0x7fedb406d750 2024-03-16T10:12:03.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedba65b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fedb4145280 0x7fedb413f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:03.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedb9659640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fedb40b2820 0x7fedb413eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54056/0 (socket says 172.21.15.17:54056) 2024-03-16T10:12:03.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedb9659640 1 -- 172.21.15.17:0/90852983 learned_addr learned my addr 172.21.15.17:0/90852983 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:12:03.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedb9e5a640 1 --2- 172.21.15.17:0/90852983 >> v2:172.21.15.174:3300/0 conn(0x7fedb406d750 0x7fedb406f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:03.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedb8e58640 1 -- 172.21.15.17:0/90852983 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4034270435 0 0) 0x7fedb406db30 con 0x7fedb40b2820 2024-03-16T10:12:03.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedb8e58640 1 -- 172.21.15.17:0/90852983 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feda0003530 con 0x7fedb40b2820 2024-03-16T10:12:03.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedba65b640 1 -- 172.21.15.17:0/90852983 >> v2:172.21.15.174:3300/0 conn(0x7fedb406d750 msgr2=0x7fedb406f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:03.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedba65b640 1 --2- 172.21.15.17:0/90852983 >> v2:172.21.15.174:3300/0 conn(0x7fedb406d750 0x7fedb406f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedba65b640 1 -- 172.21.15.17:0/90852983 >> v1:172.21.15.17:6789/0 conn(0x7fedb40b2820 legacy=0x7fedb413eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:03.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.715+0000 7fedba65b640 1 -- 172.21.15.17:0/90852983 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb40afaf0 con 0x7fedb4145280 2024-03-16T10:12:03.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedba65b640 1 --2- 172.21.15.17:0/90852983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedb4145280 0x7fedb413f5f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fedb406de30 tx=0x7fedac01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9b4100a127d181 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:03.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedb8e58640 1 -- 172.21.15.17:0/90852983 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac0a66e0 con 0x7fedb4145280 2024-03-16T10:12:03.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedb8e58640 1 -- 172.21.15.17:0/90852983 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedac0a6880 con 0x7fedb4145280 2024-03-16T10:12:03.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedb8e58640 1 -- 172.21.15.17:0/90852983 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac0a6c30 con 0x7fedb4145280 2024-03-16T10:12:03.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 -- 172.21.15.17:0/90852983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedb4145280 msgr2=0x7fedb413f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:03.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 --2- 172.21.15.17:0/90852983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedb4145280 0x7fedb413f5f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fedb406de30 tx=0x7fedac01c810 comp rx=0 tx=0).stop 2024-03-16T10:12:03.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 -- 172.21.15.17:0/90852983 shutdown_connections 2024-03-16T10:12:03.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 --2- 172.21.15.17:0/90852983 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedb4145280 0x7fedb413f5f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 --2- 172.21.15.17:0/90852983 >> v2:172.21.15.174:3300/0 conn(0x7fedb406d750 0x7fedb406f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 -- 172.21.15.17:0/90852983 >> 172.21.15.17:0/90852983 conn(0x7fedb40b55c0 msgr2=0x7fedb40b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:03.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 -- 172.21.15.17:0/90852983 shutdown_connections 2024-03-16T10:12:03.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 -- 172.21.15.17:0/90852983 wait complete. 2024-03-16T10:12:03.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 Processor -- start 2024-03-16T10:12:03.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 -- start start 2024-03-16T10:12:03.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedb406d750 0x7fedb40b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:03.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedb40b2820 0x7fedb4139110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:03.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fedb4145280 0x7fedb41dfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:03.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fedb4152550 con 0x7fedb406d750 2024-03-16T10:12:03.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fedb40b49f0 con 0x7fedb40b2820 2024-03-16T10:12:03.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedbc0be640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fedb41526d0 con 0x7fedb4145280 2024-03-16T10:12:03.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedb9e5a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedb406d750 0x7fedb40b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:03.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedb9e5a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedb406d750 0x7fedb40b0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55406/0 (socket says 172.21.15.17:55406) 2024-03-16T10:12:03.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.719+0000 7fedb9e5a640 1 -- 172.21.15.17:0/1437151232 learned_addr learned my addr 172.21.15.17:0/1437151232 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:03.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedb9659640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedb40b2820 0x7fedb4139110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:03.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedba65b640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fedb4145280 0x7fedb41dfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:03.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedb9e5a640 1 -- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fedb4145280 msgr2=0x7fedb41dfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:03.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedb9e5a640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fedb4145280 0x7fedb41dfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedb9e5a640 1 -- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedb40b2820 msgr2=0x7fedb4139110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:03.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedb9e5a640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedb40b2820 0x7fedb4139110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedb9e5a640 1 -- 172.21.15.17:0/1437151232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb40b1280 con 0x7fedb406d750 2024-03-16T10:12:03.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedba65b640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fedb4145280 0x7fedb41dfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:12:03.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedb9659640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedb40b2820 0x7fedb4139110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:12:03.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedb9e5a640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedb406d750 0x7fedb40b0680 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fedb000e930 tx=0x7fedb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:03.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedaa7fc640 1 -- 172.21.15.17:0/1437151232 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb0017070 con 0x7fedb406d750 2024-03-16T10:12:03.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedaa7fc640 1 -- 172.21.15.17:0/1437151232 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedb000ca00 con 0x7fedb406d750 2024-03-16T10:12:03.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedbc0be640 1 -- 172.21.15.17:0/1437151232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb40b0160 con 0x7fedb406d750 2024-03-16T10:12:03.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedaa7fc640 1 -- 172.21.15.17:0/1437151232 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedb000cdb0 con 0x7fedb406d750 2024-03-16T10:12:03.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedbc0be640 1 -- 172.21.15.17:0/1437151232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb414f680 con 0x7fedb406d750 2024-03-16T10:12:03.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedbc0be640 1 -- 172.21.15.17:0/1437151232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb413fd10 con 0x7fedb406d750 2024-03-16T10:12:03.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedaa7fc640 1 -- 172.21.15.17:0/1437151232 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fedb001a070 con 0x7fedb406d750 2024-03-16T10:12:03.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedaa7fc640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fed98067660 0x7fed98069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:03.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedaa7fc640 1 -- 172.21.15.17:0/1437151232 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fedb0085910 con 0x7fedb406d750 2024-03-16T10:12:03.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedb9659640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fed98067660 0x7fed98069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:03.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.723+0000 7fedb9659640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fed98067660 0x7fed98069b20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fedb4143d00 tx=0x7feda4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:03.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.727+0000 7fedaa7fc640 1 -- 172.21.15.17:0/1437151232 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fedb00569e0 con 0x7fedb406d750 2024-03-16T10:12:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.883+0000 7fedbc0be640 1 -- 172.21.15.17:0/1437151232 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fedb4068360 con 0x7fed98067660 2024-03-16T10:12:03.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.887+0000 7fedaa7fc640 1 -- 172.21.15.17:0/1437151232 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7fedb4068360 con 0x7fed98067660 2024-03-16T10:12:03.893 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:12:03.894 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-16T10:12:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.887+0000 7fedbc0be640 1 -- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fed98067660 msgr2=0x7fed98069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.887+0000 7fedbc0be640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fed98067660 0x7fed98069b20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fedb4143d00 tx=0x7feda4009040 comp rx=0 tx=0).stop 2024-03-16T10:12:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.887+0000 7fedbc0be640 1 -- 172.21.15.17:0/1437151232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedb406d750 msgr2=0x7fedb40b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.887+0000 7fedbc0be640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedb406d750 0x7fedb40b0680 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fedb000e930 tx=0x7fedb000edf0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.887+0000 7fedbc0be640 1 -- 172.21.15.17:0/1437151232 shutdown_connections 2024-03-16T10:12:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.887+0000 7fedbc0be640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fed98067660 0x7fed98069b20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.887+0000 7fedbc0be640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fedb4145280 0x7fedb41dfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.887+0000 7fedbc0be640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedb40b2820 0x7fedb4139110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.887+0000 7fedbc0be640 1 --2- 172.21.15.17:0/1437151232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fedb406d750 0x7fedb40b0680 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:03.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.887+0000 7fedbc0be640 1 -- 172.21.15.17:0/1437151232 >> 172.21.15.17:0/1437151232 conn(0x7fedb40b55c0 msgr2=0x7fedb413a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:03.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.891+0000 7fedbc0be640 1 -- 172.21.15.17:0/1437151232 shutdown_connections 2024-03-16T10:12:03.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:03.891+0000 7fedbc0be640 1 -- 172.21.15.17:0/1437151232 wait complete. 2024-03-16T10:12:03.912 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-03-16T10:12:02.036551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236768,"kb_used_data":3968,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737952,"statfs":{"total":773094113280,"available":772851662848,"internally_reserved":0,"allocated":4063232,"data_stored":2289004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002167"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":189,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-16T10:11:58.876299+0000","last_change":"2024-03-16T10:10:48.248686+0000","last_active":"2024-03-16T10:11:58.876299+0000","last_peered":"2024-03-16T10:11:58.876299+0000","last_clean":"2024-03-16T10:11:58.876299+0000","last_became_active":"2024-03-16T10:10:48.243054+0000","last_became_peered":"2024-03-16T10:10:48.243054+0000","last_unstale":"2024-03-16T10:11:58.876299+0000","last_undegraded":"2024-03-16T10:11:58.876299+0000","last_fullsized":"2024-03-16T10:11:58.876299+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_clean_scrub_stamp":"2024-03-16T10:10:46.517938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:42.906473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":946176,"data_stored":725222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":135380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":135436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":134868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":376832,"data_stored":149066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":134834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":376832,"data_stored":149066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":942080,"data_stored":725132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:12:03.913 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-16T10:12:03.913 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:12:04.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa8608f8640 1 Processor -- start 2024-03-16T10:12:04.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa8608f8640 1 -- start start 2024-03-16T10:12:04.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa8608f8640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fa85c145f30 0x7fa85c0b58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:04.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa8608f8640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa85c142540 0x7fa85c1429b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:04.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa8608f8640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa85c1454c0 con 0x7fa85c072ce0 2024-03-16T10:12:04.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa8608f8640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa85c0730c0 con 0x7fa85c142540 2024-03-16T10:12:04.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa8608f8640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fa85c145730 con 0x7fa85c145f30 2024-03-16T10:12:04.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa85ad9d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa85c142540 0x7fa85c1429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:04.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa85a59c640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fa85c072ce0 0x7fa85c06ece0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54072/0 (socket says 172.21.15.17:54072) 2024-03-16T10:12:04.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa85a59c640 1 -- 172.21.15.17:0/3580558748 learned_addr learned my addr 172.21.15.17:0/3580558748 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:12:04.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa859d9b640 1 --2- 172.21.15.17:0/3580558748 >> v2:172.21.15.174:3300/0 conn(0x7fa85c145f30 0x7fa85c0b58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:04.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa85ad9d640 1 -- 172.21.15.17:0/3580558748 >> v2:172.21.15.174:3300/0 conn(0x7fa85c145f30 msgr2=0x7fa85c0b58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:04.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa85ad9d640 1 --2- 172.21.15.17:0/3580558748 >> v2:172.21.15.174:3300/0 conn(0x7fa85c145f30 0x7fa85c0b58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa85ad9d640 1 -- 172.21.15.17:0/3580558748 >> v1:172.21.15.17:6789/0 conn(0x7fa85c072ce0 legacy=0x7fa85c06ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:04.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa85ad9d640 1 -- 172.21.15.17:0/3580558748 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c145a10 con 0x7fa85c142540 2024-03-16T10:12:04.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa85959a640 1 -- 172.21.15.17:0/3580558748 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2558754991 0 0) 0x7fa85c1454c0 con 0x7fa85c072ce0 2024-03-16T10:12:04.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.035+0000 7fa859d9b640 1 --2- 172.21.15.17:0/3580558748 >> v2:172.21.15.174:3300/0 conn(0x7fa85c145f30 0x7fa85c0b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:12:04.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa85ad9d640 1 --2- 172.21.15.17:0/3580558748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa85c142540 0x7fa85c1429b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa85c070170 tx=0x7fa85401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a87c19903509446 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:04.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa85959a640 1 -- 172.21.15.17:0/3580558748 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8540b2070 con 0x7fa85c142540 2024-03-16T10:12:04.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa85959a640 1 -- 172.21.15.17:0/3580558748 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8540a5a10 con 0x7fa85c142540 2024-03-16T10:12:04.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa85959a640 1 -- 172.21.15.17:0/3580558748 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8540a5de0 con 0x7fa85c142540 2024-03-16T10:12:04.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 -- 172.21.15.17:0/3580558748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa85c142540 msgr2=0x7fa85c1429b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:04.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 --2- 172.21.15.17:0/3580558748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa85c142540 0x7fa85c1429b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa85c070170 tx=0x7fa85401c930 comp rx=0 tx=0).stop 2024-03-16T10:12:04.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 -- 172.21.15.17:0/3580558748 shutdown_connections 2024-03-16T10:12:04.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 --2- 172.21.15.17:0/3580558748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa85c142540 0x7fa85c1429b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 --2- 172.21.15.17:0/3580558748 >> v2:172.21.15.174:3300/0 conn(0x7fa85c145f30 0x7fa85c0b58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 -- 172.21.15.17:0/3580558748 >> 172.21.15.17:0/3580558748 conn(0x7fa85c0b79d0 msgr2=0x7fa85c0b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:04.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 -- 172.21.15.17:0/3580558748 shutdown_connections 2024-03-16T10:12:04.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 -- 172.21.15.17:0/3580558748 wait complete. 2024-03-16T10:12:04.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 Processor -- start 2024-03-16T10:12:04.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 -- start start 2024-03-16T10:12:04.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa85c072ce0 0x7fa85c13cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:04.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa85c142540 0x7fa85c1dfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:04.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa85c145f30 0x7fa85c1e01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:04.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c152800 con 0x7fa85c072ce0 2024-03-16T10:12:04.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c152680 con 0x7fa85c145f30 2024-03-16T10:12:04.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa8608f8640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa85c152980 con 0x7fa85c142540 2024-03-16T10:12:04.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa85a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa85c072ce0 0x7fa85c13cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:04.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa85a59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa85c072ce0 0x7fa85c13cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55420/0 (socket says 172.21.15.17:55420) 2024-03-16T10:12:04.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa85a59c640 1 -- 172.21.15.17:0/330645522 learned_addr learned my addr 172.21.15.17:0/330645522 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:04.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa859d9b640 1 --2- 172.21.15.17:0/330645522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa85c142540 0x7fa85c1dfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:04.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa85ad9d640 1 --2- 172.21.15.17:0/330645522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa85c145f30 0x7fa85c1e01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:04.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa85a59c640 1 -- 172.21.15.17:0/330645522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa85c142540 msgr2=0x7fa85c1dfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:04.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa85a59c640 1 --2- 172.21.15.17:0/330645522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa85c142540 0x7fa85c1dfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa85a59c640 1 -- 172.21.15.17:0/330645522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa85c145f30 msgr2=0x7fa85c1e01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:04.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa85a59c640 1 --2- 172.21.15.17:0/330645522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa85c145f30 0x7fa85c1e01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa85a59c640 1 -- 172.21.15.17:0/330645522 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c144c20 con 0x7fa85c072ce0 2024-03-16T10:12:04.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.039+0000 7fa85a59c640 1 --2- 172.21.15.17:0/330645522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa85c072ce0 0x7fa85c13cf20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00c4b0 tx=0x7fa84c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:04.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.043+0000 7fa84affd640 1 -- 172.21.15.17:0/330645522 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c018070 con 0x7fa85c072ce0 2024-03-16T10:12:04.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.043+0000 7fa84affd640 1 -- 172.21.15.17:0/330645522 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84c007500 con 0x7fa85c072ce0 2024-03-16T10:12:04.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.043+0000 7fa84affd640 1 -- 172.21.15.17:0/330645522 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c004a30 con 0x7fa85c072ce0 2024-03-16T10:12:04.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.043+0000 7fa8608f8640 1 -- 172.21.15.17:0/330645522 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85c1394c0 con 0x7fa85c072ce0 2024-03-16T10:12:04.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.043+0000 7fa8608f8640 1 -- 172.21.15.17:0/330645522 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85c13e970 con 0x7fa85c072ce0 2024-03-16T10:12:04.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.043+0000 7fa8608f8640 1 -- 172.21.15.17:0/330645522 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85c06a130 con 0x7fa85c072ce0 2024-03-16T10:12:04.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.043+0000 7fa84affd640 1 -- 172.21.15.17:0/330645522 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa84c01b070 con 0x7fa85c072ce0 2024-03-16T10:12:04.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.043+0000 7fa84affd640 1 --2- 172.21.15.17:0/330645522 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa83c067660 0x7fa83c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:04.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.043+0000 7fa84affd640 1 -- 172.21.15.17:0/330645522 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fa84c084760 con 0x7fa85c072ce0 2024-03-16T10:12:04.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.043+0000 7fa859d9b640 1 --2- 172.21.15.17:0/330645522 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa83c067660 0x7fa83c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:04.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.043+0000 7fa859d9b640 1 --2- 172.21.15.17:0/330645522 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa83c067660 0x7fa83c069b20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0b5560 tx=0x7fa850008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:04.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.047+0000 7fa84affd640 1 -- 172.21.15.17:0/330645522 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa84c055770 con 0x7fa85c072ce0 2024-03-16T10:12:04.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.207+0000 7fa8608f8640 1 -- 172.21.15.17:0/330645522 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa85c1dfa00 con 0x7fa83c067660 2024-03-16T10:12:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.207+0000 7fa84affd640 1 -- 172.21.15.17:0/330645522 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13470 (secure 0 0 0) 0x7fa85c1dfa00 con 0x7fa83c067660 2024-03-16T10:12:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:12:04.215 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-16T10:12:04.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.211+0000 7fa8608f8640 1 -- 172.21.15.17:0/330645522 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa83c067660 msgr2=0x7fa83c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:04.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.211+0000 7fa8608f8640 1 --2- 172.21.15.17:0/330645522 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa83c067660 0x7fa83c069b20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0b5560 tx=0x7fa850008040 comp rx=0 tx=0).stop 2024-03-16T10:12:04.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.211+0000 7fa8608f8640 1 -- 172.21.15.17:0/330645522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa85c072ce0 msgr2=0x7fa85c13cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:04.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.211+0000 7fa8608f8640 1 --2- 172.21.15.17:0/330645522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa85c072ce0 0x7fa85c13cf20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00c4b0 tx=0x7fa84c00c970 comp rx=0 tx=0).stop 2024-03-16T10:12:04.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.211+0000 7fa8608f8640 1 -- 172.21.15.17:0/330645522 shutdown_connections 2024-03-16T10:12:04.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.211+0000 7fa8608f8640 1 --2- 172.21.15.17:0/330645522 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa83c067660 0x7fa83c069b20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.211+0000 7fa8608f8640 1 --2- 172.21.15.17:0/330645522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa85c145f30 0x7fa85c1e01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.211+0000 7fa8608f8640 1 --2- 172.21.15.17:0/330645522 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa85c142540 0x7fa85c1dfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.211+0000 7fa8608f8640 1 --2- 172.21.15.17:0/330645522 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa85c072ce0 0x7fa85c13cf20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.211+0000 7fa8608f8640 1 -- 172.21.15.17:0/330645522 >> 172.21.15.17:0/330645522 conn(0x7fa85c0b79d0 msgr2=0x7fa85c143320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:04.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.211+0000 7fa8608f8640 1 -- 172.21.15.17:0/330645522 shutdown_connections 2024-03-16T10:12:04.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.211+0000 7fa8608f8640 1 -- 172.21.15.17:0/330645522 wait complete. 2024-03-16T10:12:04.233 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-03-16T10:12:04.037024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236768,"kb_used_data":3968,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737952,"statfs":{"total":773094113280,"available":772851662848,"internally_reserved":0,"allocated":4063232,"data_stored":2289004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001702"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":189,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-16T10:11:58.876299+0000","last_change":"2024-03-16T10:10:48.248686+0000","last_active":"2024-03-16T10:11:58.876299+0000","last_peered":"2024-03-16T10:11:58.876299+0000","last_clean":"2024-03-16T10:11:58.876299+0000","last_became_active":"2024-03-16T10:10:48.243054+0000","last_became_peered":"2024-03-16T10:10:48.243054+0000","last_unstale":"2024-03-16T10:11:58.876299+0000","last_undegraded":"2024-03-16T10:11:58.876299+0000","last_fullsized":"2024-03-16T10:11:58.876299+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_clean_scrub_stamp":"2024-03-16T10:10:46.517938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:42.906473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":946176,"data_stored":725222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":135380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":135436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":356352,"data_stored":134868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":376832,"data_stored":149066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":352256,"data_stored":134834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31548,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340292,"statfs":{"total":96636764160,"available":96604459008,"internally_reserved":0,"allocated":376832,"data_stored":149066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":942080,"data_stored":725132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:12:04.234 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T10:12:04.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb516c18640 1 Processor -- start 2024-03-16T10:12:04.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb516c18640 1 -- start start 2024-03-16T10:12:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb516c18640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb51006d750 0x7fb51006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb516c18640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5100b2820 0x7fb51013eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb516c18640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb510145280 0x7fb51013f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb516c18640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fb51006db30 con 0x7fb510145280 2024-03-16T10:12:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb516c18640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fb5100b2d30 con 0x7fb51006d750 2024-03-16T10:12:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb516c18640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb51006fcf0 con 0x7fb5100b2820 2024-03-16T10:12:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb5151b5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb510145280 0x7fb51013f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:04.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb5151b5640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fb510145280 0x7fb51013f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55424/0 (socket says 172.21.15.17:55424) 2024-03-16T10:12:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb5151b5640 1 -- 172.21.15.17:0/433433515 learned_addr learned my addr 172.21.15.17:0/433433515 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb5149b4640 1 --2- 172.21.15.17:0/433433515 >> v2:172.21.15.174:3300/0 conn(0x7fb51006d750 0x7fb51006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb50ffff640 1 --2- 172.21.15.17:0/433433515 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5100b2820 0x7fb51013eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb5151b5640 1 -- 172.21.15.17:0/433433515 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5100b2820 msgr2=0x7fb51013eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb5151b5640 1 --2- 172.21.15.17:0/433433515 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5100b2820 0x7fb51013eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb5151b5640 1 -- 172.21.15.17:0/433433515 >> v2:172.21.15.174:3300/0 conn(0x7fb51006d750 msgr2=0x7fb51006f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb5151b5640 1 --2- 172.21.15.17:0/433433515 >> v2:172.21.15.174:3300/0 conn(0x7fb51006d750 0x7fb51006f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.355+0000 7fb5151b5640 1 -- 172.21.15.17:0/433433515 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51013f960 con 0x7fb510145280 2024-03-16T10:12:04.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb5151b5640 1 --2- 172.21.15.17:0/433433515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb510145280 0x7fb51013f420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb508009e50 tx=0x7fb50801c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0a0c226b1eeee6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:04.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50f7fe640 1 -- 172.21.15.17:0/433433515 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5080a5550 con 0x7fb510145280 2024-03-16T10:12:04.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50f7fe640 1 -- 172.21.15.17:0/433433515 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5080a56f0 con 0x7fb510145280 2024-03-16T10:12:04.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50f7fe640 1 -- 172.21.15.17:0/433433515 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5080a5ac0 con 0x7fb510145280 2024-03-16T10:12:04.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 -- 172.21.15.17:0/433433515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb510145280 msgr2=0x7fb51013f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:04.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 --2- 172.21.15.17:0/433433515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb510145280 0x7fb51013f420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb508009e50 tx=0x7fb50801c8d0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 -- 172.21.15.17:0/433433515 shutdown_connections 2024-03-16T10:12:04.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 --2- 172.21.15.17:0/433433515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb510145280 0x7fb51013f420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 --2- 172.21.15.17:0/433433515 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb5100b2820 0x7fb51013eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 --2- 172.21.15.17:0/433433515 >> v2:172.21.15.174:3300/0 conn(0x7fb51006d750 0x7fb51006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 -- 172.21.15.17:0/433433515 >> 172.21.15.17:0/433433515 conn(0x7fb5100b55c0 msgr2=0x7fb5100b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:04.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 -- 172.21.15.17:0/433433515 shutdown_connections 2024-03-16T10:12:04.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 -- 172.21.15.17:0/433433515 wait complete. 2024-03-16T10:12:04.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 Processor -- start 2024-03-16T10:12:04.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 -- start start 2024-03-16T10:12:04.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb51006d750 0x7fb5101e17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:04.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5100b2820 0x7fb510072c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:04.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb510145280 0x7fb5101327c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:04.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb510149390 con 0x7fb5100b2820 2024-03-16T10:12:04.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb510149210 con 0x7fb51006d750 2024-03-16T10:12:04.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb510149510 con 0x7fb510145280 2024-03-16T10:12:04.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5100b2820 0x7fb510072c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:04.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50ffff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5100b2820 0x7fb510072c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55432/0 (socket says 172.21.15.17:55432) 2024-03-16T10:12:04.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50ffff640 1 -- 172.21.15.17:0/516448611 learned_addr learned my addr 172.21.15.17:0/516448611 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:04.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb5149b4640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb51006d750 0x7fb5101e17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:04.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb5151b5640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb510145280 0x7fb5101327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:04.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50ffff640 1 -- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb510145280 msgr2=0x7fb5101327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:04.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50ffff640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb510145280 0x7fb5101327c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50ffff640 1 -- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb51006d750 msgr2=0x7fb5101e17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:04.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50ffff640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb51006d750 0x7fb5101e17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50ffff640 1 -- 172.21.15.17:0/516448611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5101368f0 con 0x7fb5100b2820 2024-03-16T10:12:04.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb5149b4640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb51006d750 0x7fb5101e17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:12:04.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50ffff640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5100b2820 0x7fb510072c40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00d7d0 tx=0x7fb4fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:04.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50d7fa640 1 -- 172.21.15.17:0/516448611 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc00b6c0 con 0x7fb5100b2820 2024-03-16T10:12:04.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50d7fa640 1 -- 172.21.15.17:0/516448611 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4fc019070 con 0x7fb5100b2820 2024-03-16T10:12:04.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb516c18640 1 -- 172.21.15.17:0/516448611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb51013dd40 con 0x7fb5100b2820 2024-03-16T10:12:04.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.359+0000 7fb50d7fa640 1 -- 172.21.15.17:0/516448611 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc00baa0 con 0x7fb5100b2820 2024-03-16T10:12:04.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.363+0000 7fb516c18640 1 -- 172.21.15.17:0/516448611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5101e1280 con 0x7fb5100b2820 2024-03-16T10:12:04.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.363+0000 7fb516c18640 1 -- 172.21.15.17:0/516448611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb510068360 con 0x7fb5100b2820 2024-03-16T10:12:04.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.363+0000 7fb50d7fa640 1 -- 172.21.15.17:0/516448611 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fb4fc00bc40 con 0x7fb5100b2820 2024-03-16T10:12:04.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.363+0000 7fb50d7fa640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb4f4067660 0x7fb4f4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:04.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.363+0000 7fb50d7fa640 1 -- 172.21.15.17:0/516448611 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fb4fc0859c0 con 0x7fb5100b2820 2024-03-16T10:12:04.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.363+0000 7fb5149b4640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb4f4067660 0x7fb4f4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:04.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.363+0000 7fb5149b4640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb4f4067660 0x7fb4f4069b20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb504009720 tx=0x7fb504008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:04.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.367+0000 7fb50d7fa640 1 -- 172.21.15.17:0/516448611 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb4fc087e20 con 0x7fb5100b2820 2024-03-16T10:12:04.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.531+0000 7fb516c18640 1 -- 172.21.15.17:0/516448611 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb510136610 con 0x7fb5100b2820 2024-03-16T10:12:04.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.531+0000 7fb50d7fa640 1 -- 172.21.15.17:0/516448611 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+13018 (secure 0 0 0) 0x7fb4fc0569d0 con 0x7fb5100b2820 2024-03-16T10:12:04.539 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:12:04.539 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":33,"fsid":"b21729d9-93de-4054-b789-790eefecfc1c","created":"2024-03-16T10:10:33.297190+0000","modified":"2024-03-16T10:12:01.651198+0000","last_up_change":"2024-03-16T10:10:44.509406+0000","last_in_change":"2024-03-16T10:10:35.591690+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T10:10:46.093125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"dd7d0093-3aa1-4b83-b1e0-b997bae8231f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6825","nonce":3540530625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6827","nonce":3540530625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6831","nonce":3540530625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":3540530625},{"type":"v1","addr":"172.21.15.17:6829","nonce":3540530625}]},"public_addr":"172.21.15.17:6825/3540530625","cluster_addr":"172.21.15.17:6827/3540530625","heartbeat_back_addr":"172.21.15.17:6831/3540530625","heartbeat_front_addr":"172.21.15.17:6829/3540530625","state":["exists","up"]},{"osd":1,"uuid":"d043363d-4176-4bcd-a4f4-aadcd5c14d97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6802","nonce":3058835082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6805","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6808","nonce":3058835082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6819","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6822","nonce":3058835082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6811","nonce":3058835082},{"type":"v1","addr":"172.21.15.17:6815","nonce":3058835082}]},"public_addr":"172.21.15.17:6802/3058835082","cluster_addr":"172.21.15.17:6808/3058835082","heartbeat_back_addr":"172.21.15.17:6822/3058835082","heartbeat_front_addr":"172.21.15.17:6815/3058835082","state":["exists","up"]},{"osd":2,"uuid":"48afbf8a-dcdd-4301-8a7e-ce26fd327df9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6801","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6804","nonce":1622136967}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6807","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6810","nonce":1622136967}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6817","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6820","nonce":1622136967}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6813","nonce":1622136967},{"type":"v1","addr":"172.21.15.17:6814","nonce":1622136967}]},"public_addr":"172.21.15.17:6804/1622136967","cluster_addr":"172.21.15.17:6810/1622136967","heartbeat_back_addr":"172.21.15.17:6820/1622136967","heartbeat_front_addr":"172.21.15.17:6814/1622136967","state":["exists","up"]},{"osd":3,"uuid":"4f84ab8d-fb79-41ba-ae35-09aeba2105f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6803","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6806","nonce":1266087291}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6809","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6812","nonce":1266087291}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6821","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6823","nonce":1266087291}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1266087291},{"type":"v1","addr":"172.21.15.17:6818","nonce":1266087291}]},"public_addr":"172.21.15.17:6806/1266087291","cluster_addr":"172.21.15.17:6812/1266087291","heartbeat_back_addr":"172.21.15.17:6823/1266087291","heartbeat_front_addr":"172.21.15.17:6818/1266087291","state":["exists","up"]},{"osd":4,"uuid":"ddb163f0-cce0-42a6-8d92-644babc32d58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6819","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6822","nonce":326911164}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6825","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6827","nonce":326911164}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6831","nonce":326911164}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":326911164},{"type":"v1","addr":"172.21.15.174:6829","nonce":326911164}]},"public_addr":"172.21.15.174:6822/326911164","cluster_addr":"172.21.15.174:6827/326911164","heartbeat_back_addr":"172.21.15.174:6831/326911164","heartbeat_front_addr":"172.21.15.174:6829/326911164","state":["exists","up"]},{"osd":5,"uuid":"6e5d81da-6610-4194-91ca-b96d05e4ae48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6801","nonce":1721037852}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6804","nonce":1721037852}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6812","nonce":1721037852}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":1721037852},{"type":"v1","addr":"172.21.15.174:6808","nonce":1721037852}]},"public_addr":"172.21.15.174:6801/1721037852","cluster_addr":"172.21.15.174:6804/1721037852","heartbeat_back_addr":"172.21.15.174:6812/1721037852","heartbeat_front_addr":"172.21.15.174:6808/1721037852","state":["exists","up"]},{"osd":6,"uuid":"0f5f0885-f9c9-4ddc-9cf0-216197b35186","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6803","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6805","nonce":4101740318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6807","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6809","nonce":4101740318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6815","nonce":4101740318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6811","nonce":4101740318},{"type":"v1","addr":"172.21.15.174:6813","nonce":4101740318}]},"public_addr":"172.21.15.174:6805/4101740318","cluster_addr":"172.21.15.174:6809/4101740318","heartbeat_back_addr":"172.21.15.174:6815/4101740318","heartbeat_front_addr":"172.21.15.174:6813/4101740318","state":["exists","up"]},{"osd":7,"uuid":"c133836d-5f03-4ecd-9876-14aa5f99b990","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6817","nonce":1083843796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6820","nonce":1083843796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6826","nonce":1083843796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6821","nonce":1083843796},{"type":"v1","addr":"172.21.15.174:6823","nonce":1083843796}]},"public_addr":"172.21.15.174:6817/1083843796","cluster_addr":"172.21.15.174:6820/1083843796","heartbeat_back_addr":"172.21.15.174:6826/1083843796","heartbeat_front_addr":"172.21.15.174:6823/1083843796","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.208703+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.175272+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.169482+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.030570+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.075121+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:40.992128+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.024515+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T10:10:41.068238+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.17:6833/578943207":"2024-03-17T10:11:58.639697+0000","172.21.15.17:6832/578943207":"2024-03-17T10:11:58.639697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T10:12:04.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.535+0000 7fb516c18640 1 -- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb4f4067660 msgr2=0x7fb4f4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:04.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.535+0000 7fb516c18640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb4f4067660 0x7fb4f4069b20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb504009720 tx=0x7fb504008040 comp rx=0 tx=0).stop 2024-03-16T10:12:04.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.535+0000 7fb516c18640 1 -- 172.21.15.17:0/516448611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5100b2820 msgr2=0x7fb510072c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:04.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.535+0000 7fb516c18640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5100b2820 0x7fb510072c40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc00d7d0 tx=0x7fb4fc00dc90 comp rx=0 tx=0).stop 2024-03-16T10:12:04.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.535+0000 7fb516c18640 1 -- 172.21.15.17:0/516448611 shutdown_connections 2024-03-16T10:12:04.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.535+0000 7fb516c18640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb4f4067660 0x7fb4f4069b20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.535+0000 7fb516c18640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb510145280 0x7fb5101327c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.535+0000 7fb516c18640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb5100b2820 0x7fb510072c40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.535+0000 7fb516c18640 1 --2- 172.21.15.17:0/516448611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb51006d750 0x7fb5101e17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:04.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.535+0000 7fb516c18640 1 -- 172.21.15.17:0/516448611 >> 172.21.15.17:0/516448611 conn(0x7fb5100b55c0 msgr2=0x7fb51013b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:04.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.535+0000 7fb516c18640 1 -- 172.21.15.17:0/516448611 shutdown_connections 2024-03-16T10:12:04.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:04.535+0000 7fb516c18640 1 -- 172.21.15.17:0/516448611 wait complete. 2024-03-16T10:12:05.559 INFO:tasks.ceph:Scrubbing osd.0 2024-03-16T10:12:05.559 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-16T10:12:05.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.691+0000 7f4e7294c640 1 Processor -- start 2024-03-16T10:12:05.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.691+0000 7f4e7294c640 1 -- start start 2024-03-16T10:12:05.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.691+0000 7f4e7294c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e6c0af140 0x7f4e6c0af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.691+0000 7f4e7294c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f4e6c143f80 0x7f4e6c0b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.691+0000 7f4e7294c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e6c06d490 con 0x7f4e6c143b20 2024-03-16T10:12:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.691+0000 7f4e7294c640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f4e6c0b0a80 con 0x7f4e6c143f80 2024-03-16T10:12:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.691+0000 7f4e7294c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4e6c0b0cb0 con 0x7f4e6c0af140 2024-03-16T10:12:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e6b7fe640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f4e6c143b20 0x7f4e6c0b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54082/0 (socket says 172.21.15.17:54082) 2024-03-16T10:12:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e6b7fe640 1 -- 172.21.15.17:0/3924934478 learned_addr learned my addr 172.21.15.17:0/3924934478 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:12:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e6bfff640 1 --2- 172.21.15.17:0/3924934478 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e6c0af140 0x7f4e6c0af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:05.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e70ee9640 1 --2- 172.21.15.17:0/3924934478 >> v2:172.21.15.174:3300/0 conn(0x7f4e6c143f80 0x7f4e6c0b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e6affd640 1 -- 172.21.15.17:0/3924934478 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2531155967 0 0) 0x7f4e6c06d490 con 0x7f4e6c143b20 2024-03-16T10:12:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e6affd640 1 -- 172.21.15.17:0/3924934478 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e54005740 con 0x7f4e6c143b20 2024-03-16T10:12:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e6affd640 1 -- 172.21.15.17:0/3924934478 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 756851794 0 0) 0x7f4e54005740 con 0x7f4e6c143b20 2024-03-16T10:12:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e6bfff640 1 -- 172.21.15.17:0/3924934478 >> v2:172.21.15.174:3300/0 conn(0x7f4e6c143f80 msgr2=0x7f4e6c0b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:05.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e6bfff640 1 --2- 172.21.15.17:0/3924934478 >> v2:172.21.15.174:3300/0 conn(0x7f4e6c143f80 0x7f4e6c0b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:05.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e6bfff640 1 -- 172.21.15.17:0/3924934478 >> v1:172.21.15.17:6789/0 conn(0x7f4e6c143b20 legacy=0x7f4e6c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:05.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e6bfff640 1 -- 172.21.15.17:0/3924934478 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6c0afa60 con 0x7f4e6c0af140 2024-03-16T10:12:05.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e6bfff640 1 --2- 172.21.15.17:0/3924934478 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e6c0af140 0x7f4e6c0af520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0097c0 tx=0x7f4e5c01c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e00b439506da8005 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:05.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e6affd640 1 -- 172.21.15.17:0/3924934478 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0b0070 con 0x7f4e6c0af140 2024-03-16T10:12:05.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e6affd640 1 -- 172.21.15.17:0/3924934478 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e5c004030 con 0x7f4e6c0af140 2024-03-16T10:12:05.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.695+0000 7f4e6affd640 1 -- 172.21.15.17:0/3924934478 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e600016a0 con 0x7f4e6c0af140 2024-03-16T10:12:05.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 -- 172.21.15.17:0/3924934478 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e6c0af140 msgr2=0x7f4e6c0af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:05.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 --2- 172.21.15.17:0/3924934478 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e6c0af140 0x7f4e6c0af520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0097c0 tx=0x7f4e5c01c380 comp rx=0 tx=0).stop 2024-03-16T10:12:05.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 -- 172.21.15.17:0/3924934478 shutdown_connections 2024-03-16T10:12:05.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 --2- 172.21.15.17:0/3924934478 >> v2:172.21.15.174:3300/0 conn(0x7f4e6c143f80 0x7f4e6c0b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:05.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 --2- 172.21.15.17:0/3924934478 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e6c0af140 0x7f4e6c0af520 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:05.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 -- 172.21.15.17:0/3924934478 >> 172.21.15.17:0/3924934478 conn(0x7f4e6c0b8e90 msgr2=0x7f4e6c0b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:05.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 -- 172.21.15.17:0/3924934478 shutdown_connections 2024-03-16T10:12:05.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 -- 172.21.15.17:0/3924934478 wait complete. 2024-03-16T10:12:05.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 Processor -- start 2024-03-16T10:12:05.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 -- start start 2024-03-16T10:12:05.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e6c0af140 0x7f4e6c0719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:05.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e6c143b20 0x7f4e6c1dbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:05.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e6c143f80 0x7f4e6c1deda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:05.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1503d0 con 0x7f4e6c143f80 2024-03-16T10:12:05.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c0aea00 con 0x7f4e6c0af140 2024-03-16T10:12:05.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f4e6c150550 con 0x7f4e6c143b20 2024-03-16T10:12:05.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e70ee9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e6c143f80 0x7f4e6c1deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:05.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e70ee9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e6c143f80 0x7f4e6c1deda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55446/0 (socket says 172.21.15.17:55446) 2024-03-16T10:12:05.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e70ee9640 1 -- 172.21.15.17:0/1618986633 learned_addr learned my addr 172.21.15.17:0/1618986633 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:05.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e6b7fe640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e6c143b20 0x7f4e6c1dbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:05.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e6bfff640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e6c0af140 0x7f4e6c0719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:05.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e70ee9640 1 -- 172.21.15.17:0/1618986633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e6c143b20 msgr2=0x7f4e6c1dbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:05.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e70ee9640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e6c143b20 0x7f4e6c1dbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:05.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e70ee9640 1 -- 172.21.15.17:0/1618986633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e6c0af140 msgr2=0x7f4e6c0719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:05.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e70ee9640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e6c0af140 0x7f4e6c0719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:05.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e70ee9640 1 -- 172.21.15.17:0/1618986633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6c072570 con 0x7f4e6c143f80 2024-03-16T10:12:05.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e6b7fe640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f4e6c143b20 0x7f4e6c1dbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:12:05.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e70ee9640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e6c143f80 0x7f4e6c1deda0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400efc0 tx=0x7f4e6400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:05.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e68ff9640 1 -- 172.21.15.17:0/1618986633 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64018020 con 0x7f4e6c143f80 2024-03-16T10:12:05.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e68ff9640 1 -- 172.21.15.17:0/1618986633 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e64014480 con 0x7f4e6c143f80 2024-03-16T10:12:05.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 -- 172.21.15.17:0/1618986633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6c070a70 con 0x7f4e6c143f80 2024-03-16T10:12:05.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e68ff9640 1 -- 172.21.15.17:0/1618986633 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64014870 con 0x7f4e6c143f80 2024-03-16T10:12:05.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 -- 172.21.15.17:0/1618986633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e6c06f600 con 0x7f4e6c143f80 2024-03-16T10:12:05.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.699+0000 7f4e7294c640 1 -- 172.21.15.17:0/1618986633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e6c1db830 con 0x7f4e6c143f80 2024-03-16T10:12:05.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.703+0000 7f4e68ff9640 1 -- 172.21.15.17:0/1618986633 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f4e6401b020 con 0x7f4e6c143f80 2024-03-16T10:12:05.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.703+0000 7f4e68ff9640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4e58067660 0x7f4e58069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.703+0000 7f4e68ff9640 1 -- 172.21.15.17:0/1618986633 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f4e64086100 con 0x7f4e6c143f80 2024-03-16T10:12:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.703+0000 7f4e68ff9640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.17:6824/3540530625,v1:172.21.15.17:6825/3540530625] conn(0x7f4e5806d520 0x7f4e5806f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.703+0000 7f4e68ff9640 1 -- 172.21.15.17:0/1618986633 --> [v2:172.21.15.17:6824/3540530625,v1:172.21.15.17:6825/3540530625] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e5806fff0 con 0x7f4e5806d520 2024-03-16T10:12:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.703+0000 7f4e68ff9640 1 -- 172.21.15.17:0/1618986633 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4e6408d050 con 0x7f4e6c143f80 2024-03-16T10:12:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.703+0000 7f4e6bfff640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4e58067660 0x7f4e58069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.703+0000 7f4e6b7fe640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.17:6824/3540530625,v1:172.21.15.17:6825/3540530625] conn(0x7f4e5806d520 0x7f4e5806f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.703+0000 7f4e6b7fe640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.17:6824/3540530625,v1:172.21.15.17:6825/3540530625] conn(0x7f4e5806d520 0x7f4e5806f940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.703+0000 7f4e6bfff640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4e58067660 0x7f4e58069b20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0ad000 tx=0x7f4e5c0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:05.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.703+0000 7f4e68ff9640 1 -- 172.21.15.17:0/1618986633 <== osd.0 v2:172.21.15.17:6824/3540530625 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4e5806fff0 con 0x7f4e5806d520 2024-03-16T10:12:05.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.715+0000 7f4e7294c640 1 -- 172.21.15.17:0/1618986633 --> [v2:172.21.15.17:6824/3540530625,v1:172.21.15.17:6825/3540530625] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4e6c068360 con 0x7f4e5806d520 2024-03-16T10:12:05.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.719+0000 7f4e68ff9640 1 -- 172.21.15.17:0/1618986633 <== osd.0 v2:172.21.15.17:6824/3540530625 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4e6c068360 con 0x7f4e5806d520 2024-03-16T10:12:05.724 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-16T10:12:05.724 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T10:12:05.724 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-16T10:12:05.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.719+0000 7f4e7294c640 1 -- 172.21.15.17:0/1618986633 >> [v2:172.21.15.17:6824/3540530625,v1:172.21.15.17:6825/3540530625] conn(0x7f4e5806d520 msgr2=0x7f4e5806f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.719+0000 7f4e7294c640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.17:6824/3540530625,v1:172.21.15.17:6825/3540530625] conn(0x7f4e5806d520 0x7f4e5806f940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.719+0000 7f4e7294c640 1 -- 172.21.15.17:0/1618986633 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4e58067660 msgr2=0x7f4e58069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.719+0000 7f4e7294c640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f4e58067660 0x7f4e58069b20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0ad000 tx=0x7f4e5c0a6000 comp rx=0 tx=0).stop 2024-03-16T10:12:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.719+0000 7f4e7294c640 1 -- 172.21.15.17:0/1618986633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e6c143f80 msgr2=0x7f4e6c1deda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.719+0000 7f4e7294c640 1 --2- 172.21.15.17:0/1618986633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e6c143f80 0x7f4e6c1deda0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400efc0 tx=0x7f4e6400c490 comp rx=0 tx=0).stop 2024-03-16T10:12:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.719+0000 7f4e70ee9640 1 -- 172.21.15.17:0/1618986633 reap_dead start 2024-03-16T10:12:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.719+0000 7f4e7294c640 1 -- 172.21.15.17:0/1618986633 shutdown_connections 2024-03-16T10:12:05.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.719+0000 7f4e7294c640 1 -- 172.21.15.17:0/1618986633 >> 172.21.15.17:0/1618986633 conn(0x7f4e6c0b8e90 msgr2=0x7f4e6c0b3180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:05.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.719+0000 7f4e7294c640 1 -- 172.21.15.17:0/1618986633 shutdown_connections 2024-03-16T10:12:05.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.719+0000 7f4e7294c640 1 -- 172.21.15.17:0/1618986633 wait complete. 2024-03-16T10:12:05.741 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-16T10:12:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd957e3a640 1 Processor -- start 2024-03-16T10:12:05.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd957e3a640 1 -- start start 2024-03-16T10:12:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd957e3a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd95006d750 0x7fd95006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd957e3a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd950145280 0x7fd95013f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd957e3a640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd95006db30 con 0x7fd9500b2820 2024-03-16T10:12:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd957e3a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd9500b2d30 con 0x7fd95006d750 2024-03-16T10:12:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd957e3a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd95006fcf0 con 0x7fd950145280 2024-03-16T10:12:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd955bd6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd95006d750 0x7fd95006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:05.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd955bd6640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd95006d750 0x7fd95006f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55462/0 (socket says 172.21.15.17:55462) 2024-03-16T10:12:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd955bd6640 1 -- 172.21.15.17:0/30776389 learned_addr learned my addr 172.21.15.17:0/30776389 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd9563d7640 1 --2- 172.21.15.17:0/30776389 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd950145280 0x7fd95013f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd955bd6640 1 -- 172.21.15.17:0/30776389 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd950145280 msgr2=0x7fd95013f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd955bd6640 1 --2- 172.21.15.17:0/30776389 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd950145280 0x7fd95013f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd955bd6640 1 -- 172.21.15.17:0/30776389 >> v1:172.21.15.174:6789/0 conn(0x7fd9500b2820 legacy=0x7fd95013eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd955bd6640 1 -- 172.21.15.17:0/30776389 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9500afaf0 con 0x7fd95006d750 2024-03-16T10:12:05.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.859+0000 7fd9563d7640 1 --2- 172.21.15.17:0/30776389 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd950145280 0x7fd95013f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:12:05.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd955bd6640 1 --2- 172.21.15.17:0/30776389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd95006d750 0x7fd95006f7b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009e50 tx=0x7fd94c01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97aa3108ee4c2eb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:05.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd954bd4640 1 -- 172.21.15.17:0/30776389 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c0a5550 con 0x7fd95006d750 2024-03-16T10:12:05.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd954bd4640 1 -- 172.21.15.17:0/30776389 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd94c0a56f0 con 0x7fd95006d750 2024-03-16T10:12:05.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd954bd4640 1 -- 172.21.15.17:0/30776389 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c0a5ac0 con 0x7fd95006d750 2024-03-16T10:12:05.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 -- 172.21.15.17:0/30776389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd95006d750 msgr2=0x7fd95006f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:05.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 --2- 172.21.15.17:0/30776389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd95006d750 0x7fd95006f7b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd94c009e50 tx=0x7fd94c01c8d0 comp rx=0 tx=0).stop 2024-03-16T10:12:05.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 -- 172.21.15.17:0/30776389 shutdown_connections 2024-03-16T10:12:05.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 --2- 172.21.15.17:0/30776389 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd950145280 0x7fd95013f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:05.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 --2- 172.21.15.17:0/30776389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd95006d750 0x7fd95006f7b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:05.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 -- 172.21.15.17:0/30776389 >> 172.21.15.17:0/30776389 conn(0x7fd9500b55c0 msgr2=0x7fd9500b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:05.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 -- 172.21.15.17:0/30776389 shutdown_connections 2024-03-16T10:12:05.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 -- 172.21.15.17:0/30776389 wait complete. 2024-03-16T10:12:05.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 Processor -- start 2024-03-16T10:12:05.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 -- start start 2024-03-16T10:12:05.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd95006d750 0x7fd950141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:05.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9500b2820 0x7fd95013c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:05.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd950145280 0x7fd95013c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:05.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd9500b0390 con 0x7fd9500b2820 2024-03-16T10:12:05.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9500b0210 con 0x7fd95006d750 2024-03-16T10:12:05.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd957e3a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd9500b0510 con 0x7fd950145280 2024-03-16T10:12:05.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd9553d5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9500b2820 0x7fd95013c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:05.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd9553d5640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9500b2820 0x7fd95013c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55470/0 (socket says 172.21.15.17:55470) 2024-03-16T10:12:05.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd9553d5640 1 -- 172.21.15.17:0/1129354742 learned_addr learned my addr 172.21.15.17:0/1129354742 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:05.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd9563d7640 1 --2- 172.21.15.17:0/1129354742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd950145280 0x7fd95013c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:05.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd955bd6640 1 --2- 172.21.15.17:0/1129354742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd95006d750 0x7fd950141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd9553d5640 1 -- 172.21.15.17:0/1129354742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd950145280 msgr2=0x7fd95013c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd9553d5640 1 --2- 172.21.15.17:0/1129354742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd950145280 0x7fd95013c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd9553d5640 1 -- 172.21.15.17:0/1129354742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd95006d750 msgr2=0x7fd950141e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd9553d5640 1 --2- 172.21.15.17:0/1129354742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd95006d750 0x7fd950141e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.863+0000 7fd9553d5640 1 -- 172.21.15.17:0/1129354742 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9500af390 con 0x7fd9500b2820 2024-03-16T10:12:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.867+0000 7fd9553d5640 1 --2- 172.21.15.17:0/1129354742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9500b2820 0x7fd95013c090 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd94000c450 tx=0x7fd94000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.867+0000 7fd9467fc640 1 -- 172.21.15.17:0/1129354742 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940012830 con 0x7fd9500b2820 2024-03-16T10:12:05.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.867+0000 7fd9467fc640 1 -- 172.21.15.17:0/1129354742 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd940017070 con 0x7fd9500b2820 2024-03-16T10:12:05.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.867+0000 7fd957e3a640 1 -- 172.21.15.17:0/1129354742 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9500af0b0 con 0x7fd9500b2820 2024-03-16T10:12:05.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.867+0000 7fd9467fc640 1 -- 172.21.15.17:0/1129354742 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940012c30 con 0x7fd9500b2820 2024-03-16T10:12:05.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.867+0000 7fd957e3a640 1 -- 172.21.15.17:0/1129354742 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd950142b60 con 0x7fd9500b2820 2024-03-16T10:12:05.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.867+0000 7fd957e3a640 1 -- 172.21.15.17:0/1129354742 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd95013dee0 con 0x7fd9500b2820 2024-03-16T10:12:05.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.867+0000 7fd9467fc640 1 -- 172.21.15.17:0/1129354742 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd94001c460 con 0x7fd9500b2820 2024-03-16T10:12:05.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.867+0000 7fd9467fc640 1 --2- 172.21.15.17:0/1129354742 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd934067660 0x7fd934069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:05.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.867+0000 7fd9467fc640 1 -- 172.21.15.17:0/1129354742 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fd940085290 con 0x7fd9500b2820 2024-03-16T10:12:05.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.867+0000 7fd955bd6640 1 --2- 172.21.15.17:0/1129354742 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd934067660 0x7fd934069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:05.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.867+0000 7fd955bd6640 1 --2- 172.21.15.17:0/1129354742 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd934067660 0x7fd934069b20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd94c0ae000 tx=0x7fd94c0a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:05.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:05.871+0000 7fd9467fc640 1 -- 172.21.15.17:0/1129354742 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd95013dee0 con 0x7fd9500b2820 2024-03-16T10:12:06.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.035+0000 7fd957e3a640 1 -- 172.21.15.17:0/1129354742 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fd950068360 con 0x7fd934067660 2024-03-16T10:12:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.035+0000 7fd9467fc640 1 -- 172.21.15.17:0/1129354742 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd950068360 con 0x7fd934067660 2024-03-16T10:12:06.041 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 0 to deep-scrub 2024-03-16T10:12:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.035+0000 7fd957e3a640 1 -- 172.21.15.17:0/1129354742 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd934067660 msgr2=0x7fd934069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.035+0000 7fd957e3a640 1 --2- 172.21.15.17:0/1129354742 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd934067660 0x7fd934069b20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd94c0ae000 tx=0x7fd94c0a7070 comp rx=0 tx=0).stop 2024-03-16T10:12:06.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.035+0000 7fd957e3a640 1 -- 172.21.15.17:0/1129354742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9500b2820 msgr2=0x7fd95013c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.035+0000 7fd957e3a640 1 --2- 172.21.15.17:0/1129354742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9500b2820 0x7fd95013c090 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd94000c450 tx=0x7fd94000c910 comp rx=0 tx=0).stop 2024-03-16T10:12:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.035+0000 7fd957e3a640 1 -- 172.21.15.17:0/1129354742 shutdown_connections 2024-03-16T10:12:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.035+0000 7fd957e3a640 1 --2- 172.21.15.17:0/1129354742 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd934067660 0x7fd934069b20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.035+0000 7fd957e3a640 1 --2- 172.21.15.17:0/1129354742 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd950145280 0x7fd95013c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.035+0000 7fd957e3a640 1 --2- 172.21.15.17:0/1129354742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9500b2820 0x7fd95013c090 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.035+0000 7fd957e3a640 1 --2- 172.21.15.17:0/1129354742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd95006d750 0x7fd950141e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.035+0000 7fd957e3a640 1 -- 172.21.15.17:0/1129354742 >> 172.21.15.17:0/1129354742 conn(0x7fd9500b55c0 msgr2=0x7fd950143e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:06.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.035+0000 7fd957e3a640 1 -- 172.21.15.17:0/1129354742 shutdown_connections 2024-03-16T10:12:06.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.035+0000 7fd957e3a640 1 -- 172.21.15.17:0/1129354742 wait complete. 2024-03-16T10:12:06.060 INFO:tasks.ceph:Scrubbing osd.1 2024-03-16T10:12:06.060 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-16T10:12:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.183+0000 7fabccd76640 1 Processor -- start 2024-03-16T10:12:06.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.183+0000 7fabccd76640 1 -- start start 2024-03-16T10:12:06.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabccd76640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fabc80b4650 0x7fabc814b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabccd76640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc80b4a30 con 0x7fabc81419f0 2024-03-16T10:12:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabccd76640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc814abe0 con 0x7fabc80b7650 2024-03-16T10:12:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabccd76640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fabc814aec0 con 0x7fabc80b4650 2024-03-16T10:12:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc6d9d640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fabc81419f0 0x7fabc8151dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54094/0 (socket says 172.21.15.17:54094) 2024-03-16T10:12:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc6d9d640 1 -- 172.21.15.17:0/3628907155 learned_addr learned my addr 172.21.15.17:0/3628907155 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:12:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc659c640 1 --2- 172.21.15.17:0/3628907155 >> v2:172.21.15.174:3300/0 conn(0x7fabc80b4650 0x7fabc814b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 -- 172.21.15.17:0/3628907155 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 842990755 0 0) 0x7fabc80b4a30 con 0x7fabc81419f0 2024-03-16T10:12:06.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 -- 172.21.15.17:0/3628907155 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fabb0003530 con 0x7fabc81419f0 2024-03-16T10:12:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 -- 172.21.15.17:0/3628907155 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3044240263 0 0) 0x7fabb0003530 con 0x7fabc81419f0 2024-03-16T10:12:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 -- 172.21.15.17:0/3628907155 >> v2:172.21.15.174:3300/0 conn(0x7fabc80b4650 msgr2=0x7fabc814b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 --2- 172.21.15.17:0/3628907155 >> v2:172.21.15.174:3300/0 conn(0x7fabc80b4650 0x7fabc814b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 -- 172.21.15.17:0/3628907155 >> v1:172.21.15.174:6789/0 conn(0x7fabc80b7650 legacy=0x7fabc81412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 -- 172.21.15.17:0/3628907155 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc814b920 con 0x7fabc81419f0 2024-03-16T10:12:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 -- 172.21.15.17:0/3628907155 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fabb8001680 con 0x7fabc81419f0 2024-03-16T10:12:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 -- 172.21.15.17:0/3628907155 >> v1:172.21.15.17:6789/0 conn(0x7fabc81419f0 legacy=0x7fabc8151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 --2- 172.21.15.17:0/3628907155 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabb00039c0 0x7fabb0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 --2- 172.21.15.17:0/3628907155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0004810 0x7fabb0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 --2- 172.21.15.17:0/3628907155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabb00051d0 0x7fabb000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 -- 172.21.15.17:0/3628907155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fabc814aec0 con 0x7fabb0004810 2024-03-16T10:12:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 -- 172.21.15.17:0/3628907155 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabb000c170 con 0x7fabb00051d0 2024-03-16T10:12:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc559a640 1 -- 172.21.15.17:0/3628907155 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fabb000c370 con 0x7fabb00039c0 2024-03-16T10:12:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc5d9b640 1 --2- 172.21.15.17:0/3628907155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0004810 0x7fabb0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc659c640 1 --2- 172.21.15.17:0/3628907155 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabb00039c0 0x7fabb0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc6d9d640 1 --2- 172.21.15.17:0/3628907155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabb00051d0 0x7fabb000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc5d9b640 1 -- 172.21.15.17:0/3628907155 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabb00039c0 msgr2=0x7fabb0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc5d9b640 1 --2- 172.21.15.17:0/3628907155 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabb00039c0 0x7fabb0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc5d9b640 1 -- 172.21.15.17:0/3628907155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabb00051d0 msgr2=0x7fabb000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc5d9b640 1 --2- 172.21.15.17:0/3628907155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabb00051d0 0x7fabb000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc6d9d640 1 -- 172.21.15.17:0/3628907155 reap_dead start 2024-03-16T10:12:06.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.187+0000 7fabc5d9b640 1 -- 172.21.15.17:0/3628907155 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb0003530 con 0x7fabb0004810 2024-03-16T10:12:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabc5d9b640 1 --2- 172.21.15.17:0/3628907155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0004810 0x7fabb0004c90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fabbc001780 tx=0x7fabbc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbd049a32007f72 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:06.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabc559a640 1 -- 172.21.15.17:0/3628907155 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc0013a0 con 0x7fabb0004810 2024-03-16T10:12:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabc559a640 1 -- 172.21.15.17:0/3628907155 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabbc0052f0 con 0x7fabb0004810 2024-03-16T10:12:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabc559a640 1 -- 172.21.15.17:0/3628907155 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc003840 con 0x7fabb0004810 2024-03-16T10:12:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 -- 172.21.15.17:0/3628907155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0004810 msgr2=0x7fabb0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 --2- 172.21.15.17:0/3628907155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0004810 0x7fabb0004c90 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fabbc001780 tx=0x7fabbc01c400 comp rx=0 tx=0).stop 2024-03-16T10:12:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 -- 172.21.15.17:0/3628907155 shutdown_connections 2024-03-16T10:12:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 --2- 172.21.15.17:0/3628907155 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabb0004810 0x7fabb0004c90 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 -- 172.21.15.17:0/3628907155 >> 172.21.15.17:0/3628907155 conn(0x7fabc80bccd0 msgr2=0x7fabc80bd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 -- 172.21.15.17:0/3628907155 shutdown_connections 2024-03-16T10:12:06.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 -- 172.21.15.17:0/3628907155 wait complete. 2024-03-16T10:12:06.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 Processor -- start 2024-03-16T10:12:06.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 -- start start 2024-03-16T10:12:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabb0004810 0x7fabc8148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabc8138740 0x7fabc8138bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc81e0b40 0x7fabc81e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fabc80b94a0 con 0x7fabc8138740 2024-03-16T10:12:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabc80b9320 con 0x7fabc81e0b40 2024-03-16T10:12:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabccd76640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fabc80b59f0 con 0x7fabb0004810 2024-03-16T10:12:06.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabc5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabc8138740 0x7fabc8138bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabc5d9b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabc8138740 0x7fabc8138bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55494/0 (socket says 172.21.15.17:55494) 2024-03-16T10:12:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabc5d9b640 1 -- 172.21.15.17:0/1724043732 learned_addr learned my addr 172.21.15.17:0/1724043732 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabc6d9d640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc81e0b40 0x7fabc81e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.191+0000 7fabc659c640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabb0004810 0x7fabc8148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabc5d9b640 1 -- 172.21.15.17:0/1724043732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabb0004810 msgr2=0x7fabc8148cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabc5d9b640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fabb0004810 0x7fabc8148cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabc5d9b640 1 -- 172.21.15.17:0/1724043732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc81e0b40 msgr2=0x7fabc81e0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabc5d9b640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc81e0b40 0x7fabc81e0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabc5d9b640 1 -- 172.21.15.17:0/1724043732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc81e1500 con 0x7fabc8138740 2024-03-16T10:12:06.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabc6d9d640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabc81e0b40 0x7fabc81e0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:12:06.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabc5d9b640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabc8138740 0x7fabc8138bc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fabbc01c230 tx=0x7fabb0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:06.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabb6ffd640 1 -- 172.21.15.17:0/1724043732 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc007a40 con 0x7fabc8138740 2024-03-16T10:12:06.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabb6ffd640 1 -- 172.21.15.17:0/1724043732 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabbc0ba070 con 0x7fabc8138740 2024-03-16T10:12:06.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabb6ffd640 1 -- 172.21.15.17:0/1724043732 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc0054a0 con 0x7fabc8138740 2024-03-16T10:12:06.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabccd76640 1 -- 172.21.15.17:0/1724043732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc8149390 con 0x7fabc8138740 2024-03-16T10:12:06.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabccd76640 1 -- 172.21.15.17:0/1724043732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc80bc570 con 0x7fabc8138740 2024-03-16T10:12:06.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabccd76640 1 -- 172.21.15.17:0/1724043732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fabc8068360 con 0x7fabc8138740 2024-03-16T10:12:06.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabb6ffd640 1 -- 172.21.15.17:0/1724043732 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fabbc007030 con 0x7fabc8138740 2024-03-16T10:12:06.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabb6ffd640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7faba8067660 0x7faba8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabb6ffd640 1 -- 172.21.15.17:0/1724043732 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fabbc1200c0 con 0x7fabc8138740 2024-03-16T10:12:06.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabb6ffd640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.17:6800/3058835082,v1:172.21.15.17:6802/3058835082] conn(0x7faba806d520 0x7faba806f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabb6ffd640 1 -- 172.21.15.17:0/1724043732 --> [v2:172.21.15.17:6800/3058835082,v1:172.21.15.17:6802/3058835082] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fabbc00bf30 con 0x7faba806d520 2024-03-16T10:12:06.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabb6ffd640 1 -- 172.21.15.17:0/1724043732 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fabbc120470 con 0x7fabc8138740 2024-03-16T10:12:06.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabc659c640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7faba8067660 0x7faba8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabc6d9d640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.17:6800/3058835082,v1:172.21.15.17:6802/3058835082] conn(0x7faba806d520 0x7faba806f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabc6d9d640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.17:6800/3058835082,v1:172.21.15.17:6802/3058835082] conn(0x7faba806d520 0x7faba806f940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:06.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.195+0000 7fabc659c640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7faba8067660 0x7faba8069b20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fabc0014510 tx=0x7fabc000b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:06.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.199+0000 7fabb6ffd640 1 -- 172.21.15.17:0/1724043732 <== osd.1 v2:172.21.15.17:6800/3058835082 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fabbc00bf30 con 0x7faba806d520 2024-03-16T10:12:06.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.211+0000 7fabccd76640 1 -- 172.21.15.17:0/1724043732 --> [v2:172.21.15.17:6800/3058835082,v1:172.21.15.17:6802/3058835082] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fabc81e0970 con 0x7faba806d520 2024-03-16T10:12:06.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.211+0000 7fabb6ffd640 1 -- 172.21.15.17:0/1724043732 <== osd.1 v2:172.21.15.17:6800/3058835082 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fabc81e0970 con 0x7faba806d520 2024-03-16T10:12:06.218 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-16T10:12:06.218 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T10:12:06.218 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-16T10:12:06.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.211+0000 7fabccd76640 1 -- 172.21.15.17:0/1724043732 >> [v2:172.21.15.17:6800/3058835082,v1:172.21.15.17:6802/3058835082] conn(0x7faba806d520 msgr2=0x7faba806f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.211+0000 7fabccd76640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.17:6800/3058835082,v1:172.21.15.17:6802/3058835082] conn(0x7faba806d520 0x7faba806f940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.211+0000 7fabccd76640 1 -- 172.21.15.17:0/1724043732 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7faba8067660 msgr2=0x7faba8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.211+0000 7fabccd76640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7faba8067660 0x7faba8069b20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fabc0014510 tx=0x7fabc000b040 comp rx=0 tx=0).stop 2024-03-16T10:12:06.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.211+0000 7fabccd76640 1 -- 172.21.15.17:0/1724043732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabc8138740 msgr2=0x7fabc8138bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.211+0000 7fabccd76640 1 --2- 172.21.15.17:0/1724043732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabc8138740 0x7fabc8138bc0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fabbc01c230 tx=0x7fabb0005840 comp rx=0 tx=0).stop 2024-03-16T10:12:06.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.211+0000 7fabc6d9d640 1 -- 172.21.15.17:0/1724043732 reap_dead start 2024-03-16T10:12:06.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.211+0000 7fabccd76640 1 -- 172.21.15.17:0/1724043732 shutdown_connections 2024-03-16T10:12:06.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.211+0000 7fabccd76640 1 -- 172.21.15.17:0/1724043732 >> 172.21.15.17:0/1724043732 conn(0x7fabc80bccd0 msgr2=0x7fabc813dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:06.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.211+0000 7fabccd76640 1 -- 172.21.15.17:0/1724043732 shutdown_connections 2024-03-16T10:12:06.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.211+0000 7fabccd76640 1 -- 172.21.15.17:0/1724043732 wait complete. 2024-03-16T10:12:06.234 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-16T10:12:06.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.339+0000 7f822fb8b640 1 Processor -- start 2024-03-16T10:12:06.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822fb8b640 1 -- start start 2024-03-16T10:12:06.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822fb8b640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f82280b2820 0x7f822813eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822fb8b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8228145280 0x7f822813f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822fb8b640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f822806db30 con 0x7f822806d750 2024-03-16T10:12:06.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822fb8b640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f82280b2d30 con 0x7f8228145280 2024-03-16T10:12:06.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822fb8b640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f822813f960 con 0x7f82280b2820 2024-03-16T10:12:06.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822e128640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8228145280 0x7f822813f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822d927640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f822806d750 0x7f822806f7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54108/0 (socket says 172.21.15.17:54108) 2024-03-16T10:12:06.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822e128640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8228145280 0x7f822813f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55506/0 (socket says 172.21.15.17:55506) 2024-03-16T10:12:06.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822d927640 1 -- 172.21.15.17:0/436471593 learned_addr learned my addr 172.21.15.17:0/436471593 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:12:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822d126640 1 --2- 172.21.15.17:0/436471593 >> v2:172.21.15.174:3300/0 conn(0x7f82280b2820 0x7f822813eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822e128640 1 -- 172.21.15.17:0/436471593 >> v2:172.21.15.174:3300/0 conn(0x7f82280b2820 msgr2=0x7f822813eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822e128640 1 --2- 172.21.15.17:0/436471593 >> v2:172.21.15.174:3300/0 conn(0x7f82280b2820 0x7f822813eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822e128640 1 -- 172.21.15.17:0/436471593 >> v1:172.21.15.17:6789/0 conn(0x7f822806d750 legacy=0x7f822806f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822e128640 1 -- 172.21.15.17:0/436471593 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82280afaf0 con 0x7f8228145280 2024-03-16T10:12:06.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822e128640 1 --2- 172.21.15.17:0/436471593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8228145280 0x7f822813f420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8228070780 tx=0x7f822001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ee492464509657 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:06.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822c925640 1 -- 172.21.15.17:0/436471593 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82200a66e0 con 0x7f8228145280 2024-03-16T10:12:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822c925640 1 -- 172.21.15.17:0/436471593 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82200a6880 con 0x7f8228145280 2024-03-16T10:12:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.343+0000 7f822c925640 1 -- 172.21.15.17:0/436471593 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82200a6c50 con 0x7f8228145280 2024-03-16T10:12:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 -- 172.21.15.17:0/436471593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8228145280 msgr2=0x7f822813f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 --2- 172.21.15.17:0/436471593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8228145280 0x7f822813f420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f8228070780 tx=0x7f822001c810 comp rx=0 tx=0).stop 2024-03-16T10:12:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 -- 172.21.15.17:0/436471593 shutdown_connections 2024-03-16T10:12:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 --2- 172.21.15.17:0/436471593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8228145280 0x7f822813f420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 --2- 172.21.15.17:0/436471593 >> v2:172.21.15.174:3300/0 conn(0x7f82280b2820 0x7f822813eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 -- 172.21.15.17:0/436471593 >> 172.21.15.17:0/436471593 conn(0x7f82280b55c0 msgr2=0x7f82280b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:06.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 -- 172.21.15.17:0/436471593 shutdown_connections 2024-03-16T10:12:06.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 -- 172.21.15.17:0/436471593 wait complete. 2024-03-16T10:12:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 Processor -- start 2024-03-16T10:12:06.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 -- start start 2024-03-16T10:12:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f822806d750 0x7f82281ddca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82280b2820 0x7f822813da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8228145280 0x7f82281e1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8228152550 con 0x7f822806d750 2024-03-16T10:12:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f82280b49f0 con 0x7f8228145280 2024-03-16T10:12:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f82281526d0 con 0x7f82280b2820 2024-03-16T10:12:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822d927640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f822806d750 0x7f82281ddca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822d927640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f822806d750 0x7f82281ddca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55518/0 (socket says 172.21.15.17:55518) 2024-03-16T10:12:06.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822d927640 1 -- 172.21.15.17:0/743002099 learned_addr learned my addr 172.21.15.17:0/743002099 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822d126640 1 --2- 172.21.15.17:0/743002099 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82280b2820 0x7f822813da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822e128640 1 --2- 172.21.15.17:0/743002099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8228145280 0x7f82281e1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822d927640 1 -- 172.21.15.17:0/743002099 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82280b2820 msgr2=0x7f822813da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822d927640 1 --2- 172.21.15.17:0/743002099 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82280b2820 0x7f822813da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822d927640 1 -- 172.21.15.17:0/743002099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8228145280 msgr2=0x7f82281e1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822d927640 1 --2- 172.21.15.17:0/743002099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8228145280 0x7f82281e1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822d927640 1 -- 172.21.15.17:0/743002099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82281de330 con 0x7f822806d750 2024-03-16T10:12:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822d927640 1 --2- 172.21.15.17:0/743002099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f822806d750 0x7f82281ddca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f822400c420 tx=0x7f822400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:06.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f821e7fc640 1 -- 172.21.15.17:0/743002099 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224017070 con 0x7f822806d750 2024-03-16T10:12:06.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f821e7fc640 1 -- 172.21.15.17:0/743002099 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8224005cf0 con 0x7f822806d750 2024-03-16T10:12:06.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 -- 172.21.15.17:0/743002099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82281de610 con 0x7f822806d750 2024-03-16T10:12:06.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f821e7fc640 1 -- 172.21.15.17:0/743002099 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82240072f0 con 0x7f822806d750 2024-03-16T10:12:06.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 -- 172.21.15.17:0/743002099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82281402e0 con 0x7f822806d750 2024-03-16T10:12:06.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.347+0000 7f822fb8b640 1 -- 172.21.15.17:0/743002099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8228068360 con 0x7f822806d750 2024-03-16T10:12:06.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.351+0000 7f821e7fc640 1 -- 172.21.15.17:0/743002099 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f822401a070 con 0x7f822806d750 2024-03-16T10:12:06.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.351+0000 7f821e7fc640 1 --2- 172.21.15.17:0/743002099 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f820c067660 0x7f820c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.351+0000 7f821e7fc640 1 -- 172.21.15.17:0/743002099 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f8224084930 con 0x7f822806d750 2024-03-16T10:12:06.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.351+0000 7f822d126640 1 --2- 172.21.15.17:0/743002099 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f820c067660 0x7f820c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.351+0000 7f822d126640 1 --2- 172.21.15.17:0/743002099 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f820c067660 0x7f820c069b20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f82280b2790 tx=0x7f8218008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:06.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.355+0000 7f821e7fc640 1 -- 172.21.15.17:0/743002099 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8224055940 con 0x7f822806d750 2024-03-16T10:12:06.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.515+0000 7f822fb8b640 1 -- 172.21.15.17:0/743002099 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f822813f740 con 0x7f820c067660 2024-03-16T10:12:06.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.515+0000 7f821e7fc640 1 -- 172.21.15.17:0/743002099 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f822813f740 con 0x7f820c067660 2024-03-16T10:12:06.522 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 1 to deep-scrub 2024-03-16T10:12:06.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.519+0000 7f822fb8b640 1 -- 172.21.15.17:0/743002099 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f820c067660 msgr2=0x7f820c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.519+0000 7f822fb8b640 1 --2- 172.21.15.17:0/743002099 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f820c067660 0x7f820c069b20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f82280b2790 tx=0x7f8218008040 comp rx=0 tx=0).stop 2024-03-16T10:12:06.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.519+0000 7f822fb8b640 1 -- 172.21.15.17:0/743002099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f822806d750 msgr2=0x7f82281ddca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.519+0000 7f822fb8b640 1 --2- 172.21.15.17:0/743002099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f822806d750 0x7f82281ddca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f822400c420 tx=0x7f822400c8e0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.519+0000 7f822fb8b640 1 -- 172.21.15.17:0/743002099 shutdown_connections 2024-03-16T10:12:06.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.519+0000 7f822fb8b640 1 --2- 172.21.15.17:0/743002099 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f820c067660 0x7f820c069b20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.519+0000 7f822fb8b640 1 --2- 172.21.15.17:0/743002099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8228145280 0x7f82281e1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.519+0000 7f822fb8b640 1 --2- 172.21.15.17:0/743002099 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f82280b2820 0x7f822813da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.519+0000 7f822fb8b640 1 --2- 172.21.15.17:0/743002099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f822806d750 0x7f82281ddca0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.519+0000 7f822fb8b640 1 -- 172.21.15.17:0/743002099 >> 172.21.15.17:0/743002099 conn(0x7f82280b55c0 msgr2=0x7f82280b3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:06.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.519+0000 7f822fb8b640 1 -- 172.21.15.17:0/743002099 shutdown_connections 2024-03-16T10:12:06.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.519+0000 7f822fb8b640 1 -- 172.21.15.17:0/743002099 wait complete. 2024-03-16T10:12:06.540 INFO:tasks.ceph:Scrubbing osd.2 2024-03-16T10:12:06.540 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-16T10:12:06.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.659+0000 7f01ca76c640 1 Processor -- start 2024-03-16T10:12:06.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.659+0000 7f01ca76c640 1 -- start start 2024-03-16T10:12:06.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.659+0000 7f01ca76c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c40b5160 0x7f01c40ae720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.659+0000 7f01ca76c640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f01c40af8a0 0x7f01c40729f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.659+0000 7f01ca76c640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c40af090 con 0x7f01c406c930 2024-03-16T10:12:06.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.659+0000 7f01ca76c640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f01c406cd10 con 0x7f01c40af8a0 2024-03-16T10:12:06.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.659+0000 7f01ca76c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f01c40b01e0 con 0x7f01c40b5160 2024-03-16T10:12:06.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.659+0000 7f01c3fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f01c406c930 0x7f01c40b3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54114/0 (socket says 172.21.15.17:54114) 2024-03-16T10:12:06.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.659+0000 7f01c3fff640 1 -- 172.21.15.17:0/2527665709 learned_addr learned my addr 172.21.15.17:0/2527665709 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:12:06.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.659+0000 7f01c8d09640 1 --2- 172.21.15.17:0/2527665709 >> v2:172.21.15.174:3300/0 conn(0x7f01c40af8a0 0x7f01c40729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.659+0000 7f01c37fe640 1 --2- 172.21.15.17:0/2527665709 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c40b5160 0x7f01c40ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.659+0000 7f01c2ffd640 1 -- 172.21.15.17:0/2527665709 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 964207156 0 0) 0x7f01ac001120 con 0x7f01c406c930 2024-03-16T10:12:06.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.659+0000 7f01c2ffd640 1 -- 172.21.15.17:0/2527665709 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01b0003530 con 0x7f01c406c930 2024-03-16T10:12:06.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 -- 172.21.15.17:0/2527665709 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2350917824 0 0) 0x7f01b0003530 con 0x7f01c406c930 2024-03-16T10:12:06.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 -- 172.21.15.17:0/2527665709 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c40b5160 msgr2=0x7f01c40ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:06.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 --2- 172.21.15.17:0/2527665709 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c40b5160 0x7f01c40ae720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 -- 172.21.15.17:0/2527665709 >> v2:172.21.15.174:3300/0 conn(0x7f01c40af8a0 msgr2=0x7f01c40729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:06.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 --2- 172.21.15.17:0/2527665709 >> v2:172.21.15.174:3300/0 conn(0x7f01c40af8a0 0x7f01c40729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 -- 172.21.15.17:0/2527665709 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c40aedb0 con 0x7f01c406c930 2024-03-16T10:12:06.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 -- 172.21.15.17:0/2527665709 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f01ac001850 con 0x7f01c406c930 2024-03-16T10:12:06.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 -- 172.21.15.17:0/2527665709 >> v1:172.21.15.17:6789/0 conn(0x7f01c406c930 legacy=0x7f01c40b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 --2- 172.21.15.17:0/2527665709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01b00039c0 0x7f01b0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 --2- 172.21.15.17:0/2527665709 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c40b5160 0x7f01b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 --2- 172.21.15.17:0/2527665709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01b0004d50 0x7f01b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 -- 172.21.15.17:0/2527665709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f01c406cd10 con 0x7f01b0004d50 2024-03-16T10:12:06.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 -- 172.21.15.17:0/2527665709 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01c40b01e0 con 0x7f01b00039c0 2024-03-16T10:12:06.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 -- 172.21.15.17:0/2527665709 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f01b000ba00 con 0x7f01c40b5160 2024-03-16T10:12:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c8d09640 1 --2- 172.21.15.17:0/2527665709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01b0004d50 0x7f01b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c37fe640 1 --2- 172.21.15.17:0/2527665709 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c40b5160 0x7f01b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c3fff640 1 --2- 172.21.15.17:0/2527665709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01b00039c0 0x7f01b0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c8d09640 1 -- 172.21.15.17:0/2527665709 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c40b5160 msgr2=0x7f01b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c8d09640 1 --2- 172.21.15.17:0/2527665709 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c40b5160 0x7f01b0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c8d09640 1 -- 172.21.15.17:0/2527665709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01b00039c0 msgr2=0x7f01b0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c8d09640 1 --2- 172.21.15.17:0/2527665709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01b00039c0 0x7f01b0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c8d09640 1 -- 172.21.15.17:0/2527665709 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b0003530 con 0x7f01b0004d50 2024-03-16T10:12:06.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c37fe640 1 --2- 172.21.15.17:0/2527665709 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c40b5160 0x7f01b0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:12:06.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c8d09640 1 --2- 172.21.15.17:0/2527665709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01b0004d50 0x7f01b00051d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f01c406df30 tx=0x7f01bc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1181e4b08b92f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:06.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 -- 172.21.15.17:0/2527665709 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc0b2070 con 0x7f01b0004d50 2024-03-16T10:12:06.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 -- 172.21.15.17:0/2527665709 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01bc0a5c70 con 0x7f01b0004d50 2024-03-16T10:12:06.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c2ffd640 1 -- 172.21.15.17:0/2527665709 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc009c10 con 0x7f01b0004d50 2024-03-16T10:12:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01ca76c640 1 -- 172.21.15.17:0/2527665709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01b0004d50 msgr2=0x7f01b00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01ca76c640 1 --2- 172.21.15.17:0/2527665709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01b0004d50 0x7f01b00051d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f01c406df30 tx=0x7f01bc01cc60 comp rx=0 tx=0).stop 2024-03-16T10:12:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01c8d09640 1 -- 172.21.15.17:0/2527665709 reap_dead start 2024-03-16T10:12:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01ca76c640 1 -- 172.21.15.17:0/2527665709 shutdown_connections 2024-03-16T10:12:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01ca76c640 1 -- 172.21.15.17:0/2527665709 >> 172.21.15.17:0/2527665709 conn(0x7f01c40b8e90 msgr2=0x7f01c40b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01ca76c640 1 -- 172.21.15.17:0/2527665709 shutdown_connections 2024-03-16T10:12:06.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.663+0000 7f01ca76c640 1 -- 172.21.15.17:0/2527665709 wait complete. 2024-03-16T10:12:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01ca76c640 1 Processor -- start 2024-03-16T10:12:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01ca76c640 1 -- start start 2024-03-16T10:12:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01ca76c640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c41e0690 0x7f01c41e0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01ca76c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c41e0fb0 0x7f01c4146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01ca76c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c4123ef0 0x7f01c4147050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01ca76c640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f01c40ae8a0 con 0x7f01c41e0fb0 2024-03-16T10:12:06.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01ca76c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01c40ae720 con 0x7f01c4123ef0 2024-03-16T10:12:06.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01ca76c640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f01c40aea20 con 0x7f01c41e0690 2024-03-16T10:12:06.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c41e0fb0 0x7f01c4146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c37fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c41e0fb0 0x7f01c4146b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55534/0 (socket says 172.21.15.17:55534) 2024-03-16T10:12:06.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c37fe640 1 -- 172.21.15.17:0/345841170 learned_addr learned my addr 172.21.15.17:0/345841170 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:06.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c8d09640 1 --2- 172.21.15.17:0/345841170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c4123ef0 0x7f01c4147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c3fff640 1 --2- 172.21.15.17:0/345841170 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c41e0690 0x7f01c41e0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c37fe640 1 -- 172.21.15.17:0/345841170 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c41e0690 msgr2=0x7f01c41e0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c37fe640 1 --2- 172.21.15.17:0/345841170 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f01c41e0690 0x7f01c41e0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c37fe640 1 -- 172.21.15.17:0/345841170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c4123ef0 msgr2=0x7f01c4147050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c37fe640 1 --2- 172.21.15.17:0/345841170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01c4123ef0 0x7f01c4147050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c37fe640 1 -- 172.21.15.17:0/345841170 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c41262e0 con 0x7f01c41e0fb0 2024-03-16T10:12:06.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c37fe640 1 --2- 172.21.15.17:0/345841170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c41e0fb0 0x7f01c4146b10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f01b40116d0 tx=0x7f01b4011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:06.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c0ff9640 1 -- 172.21.15.17:0/345841170 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4004220 con 0x7f01c41e0fb0 2024-03-16T10:12:06.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c0ff9640 1 -- 172.21.15.17:0/345841170 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b40043c0 con 0x7f01c41e0fb0 2024-03-16T10:12:06.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01ca76c640 1 -- 172.21.15.17:0/345841170 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c4147590 con 0x7f01c41e0fb0 2024-03-16T10:12:06.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c0ff9640 1 -- 172.21.15.17:0/345841170 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b400f9c0 con 0x7f01c41e0fb0 2024-03-16T10:12:06.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01ca76c640 1 -- 172.21.15.17:0/345841170 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c41247e0 con 0x7f01c41e0fb0 2024-03-16T10:12:06.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01ca76c640 1 -- 172.21.15.17:0/345841170 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f01c4124a20 con 0x7f01c41e0fb0 2024-03-16T10:12:06.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c0ff9640 1 -- 172.21.15.17:0/345841170 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f01b400fb60 con 0x7f01c41e0fb0 2024-03-16T10:12:06.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.667+0000 7f01c0ff9640 1 --2- 172.21.15.17:0/345841170 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f01a4067660 0x7f01a4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.671+0000 7f01c0ff9640 1 -- 172.21.15.17:0/345841170 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f01b4086350 con 0x7f01c41e0fb0 2024-03-16T10:12:06.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.671+0000 7f01c0ff9640 1 --2- 172.21.15.17:0/345841170 >> [v2:172.21.15.17:6801/1622136967,v1:172.21.15.17:6804/1622136967] conn(0x7f01a406d520 0x7f01a406f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.671+0000 7f01c0ff9640 1 -- 172.21.15.17:0/345841170 --> [v2:172.21.15.17:6801/1622136967,v1:172.21.15.17:6804/1622136967] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f01b401ee50 con 0x7f01a406d520 2024-03-16T10:12:06.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.671+0000 7f01c0ff9640 1 -- 172.21.15.17:0/345841170 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f01b4086700 con 0x7f01c41e0fb0 2024-03-16T10:12:06.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.671+0000 7f01c8d09640 1 --2- 172.21.15.17:0/345841170 >> [v2:172.21.15.17:6801/1622136967,v1:172.21.15.17:6804/1622136967] conn(0x7f01a406d520 0x7f01a406f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.671+0000 7f01c3fff640 1 --2- 172.21.15.17:0/345841170 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f01a4067660 0x7f01a4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.671+0000 7f01c8d09640 1 --2- 172.21.15.17:0/345841170 >> [v2:172.21.15.17:6801/1622136967,v1:172.21.15.17:6804/1622136967] conn(0x7f01a406d520 0x7f01a406f940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:06.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.671+0000 7f01c3fff640 1 --2- 172.21.15.17:0/345841170 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f01a4067660 0x7f01a4069b20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f01ac001ec0 tx=0x7f01ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:06.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.671+0000 7f01c0ff9640 1 -- 172.21.15.17:0/345841170 <== osd.2 v2:172.21.15.17:6801/1622136967 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f01b401ee50 con 0x7f01a406d520 2024-03-16T10:12:06.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.683+0000 7f01ca76c640 1 -- 172.21.15.17:0/345841170 --> [v2:172.21.15.17:6801/1622136967,v1:172.21.15.17:6804/1622136967] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f01c4068360 con 0x7f01a406d520 2024-03-16T10:12:06.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.683+0000 7f01c0ff9640 1 -- 172.21.15.17:0/345841170 <== osd.2 v2:172.21.15.17:6801/1622136967 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f01c4068360 con 0x7f01a406d520 2024-03-16T10:12:06.691 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-16T10:12:06.691 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T10:12:06.691 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-16T10:12:06.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.683+0000 7f01ca76c640 1 -- 172.21.15.17:0/345841170 >> [v2:172.21.15.17:6801/1622136967,v1:172.21.15.17:6804/1622136967] conn(0x7f01a406d520 msgr2=0x7f01a406f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.683+0000 7f01ca76c640 1 --2- 172.21.15.17:0/345841170 >> [v2:172.21.15.17:6801/1622136967,v1:172.21.15.17:6804/1622136967] conn(0x7f01a406d520 0x7f01a406f940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.683+0000 7f01ca76c640 1 -- 172.21.15.17:0/345841170 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f01a4067660 msgr2=0x7f01a4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.683+0000 7f01ca76c640 1 --2- 172.21.15.17:0/345841170 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f01a4067660 0x7f01a4069b20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f01ac001ec0 tx=0x7f01ac009040 comp rx=0 tx=0).stop 2024-03-16T10:12:06.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.683+0000 7f01ca76c640 1 -- 172.21.15.17:0/345841170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c41e0fb0 msgr2=0x7f01c4146b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.683+0000 7f01ca76c640 1 --2- 172.21.15.17:0/345841170 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f01c41e0fb0 0x7f01c4146b10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f01b40116d0 tx=0x7f01b4011b90 comp rx=0 tx=0).stop 2024-03-16T10:12:06.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.683+0000 7f01c8d09640 1 -- 172.21.15.17:0/345841170 reap_dead start 2024-03-16T10:12:06.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.683+0000 7f01ca76c640 1 -- 172.21.15.17:0/345841170 shutdown_connections 2024-03-16T10:12:06.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.683+0000 7f01ca76c640 1 -- 172.21.15.17:0/345841170 >> 172.21.15.17:0/345841170 conn(0x7f01c40b8e90 msgr2=0x7f01c40b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:06.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.683+0000 7f01ca76c640 1 -- 172.21.15.17:0/345841170 shutdown_connections 2024-03-16T10:12:06.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.683+0000 7f01ca76c640 1 -- 172.21.15.17:0/345841170 wait complete. 2024-03-16T10:12:06.706 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-16T10:12:06.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b758d3640 1 Processor -- start 2024-03-16T10:12:06.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b758d3640 1 -- start start 2024-03-16T10:12:06.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b758d3640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9b700b2830 0x7f9b7014a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b758d3640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b700b2ce0 0x7f9b7014af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b758d3640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b7006db30 con 0x7f9b7006d750 2024-03-16T10:12:06.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b758d3640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f9b700b5170 con 0x7f9b700b2830 2024-03-16T10:12:06.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b758d3640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9b700b4b20 con 0x7f9b700b2ce0 2024-03-16T10:12:06.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b6f7fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b700b2ce0 0x7f9b7014af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b6e7fc640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f9b700b2830 0x7f9b7014a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b6f7fe640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b700b2ce0 0x7f9b7014af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.17:49180/0 (socket says 172.21.15.17:49180) 2024-03-16T10:12:06.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b6f7fe640 1 -- 172.21.15.17:0/1098118423 learned_addr learned my addr 172.21.15.17:0/1098118423 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:06.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b6f7fe640 1 -- 172.21.15.17:0/1098118423 >> v2:172.21.15.174:3300/0 conn(0x7f9b700b2830 msgr2=0x7f9b7014a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:06.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b6f7fe640 1 --2- 172.21.15.17:0/1098118423 >> v2:172.21.15.174:3300/0 conn(0x7f9b700b2830 0x7f9b7014a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b6f7fe640 1 -- 172.21.15.17:0/1098118423 >> v1:172.21.15.174:6789/0 conn(0x7f9b7006d750 legacy=0x7f9b700ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b6f7fe640 1 -- 172.21.15.17:0/1098118423 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b700b4e00 con 0x7f9b700b2ce0 2024-03-16T10:12:06.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.811+0000 7f9b6e7fc640 1 --2- 172.21.15.17:0/1098118423 >> v2:172.21.15.174:3300/0 conn(0x7f9b700b2830 0x7f9b7014a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:12:06.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b6f7fe640 1 --2- 172.21.15.17:0/1098118423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b700b2ce0 0x7f9b7014af00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009e50 tx=0x7f9b6801c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cad204f8194b69b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:06.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b6dffb640 1 -- 172.21.15.17:0/1098118423 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b680a5770 con 0x7f9b700b2ce0 2024-03-16T10:12:06.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b6dffb640 1 -- 172.21.15.17:0/1098118423 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b680a5910 con 0x7f9b700b2ce0 2024-03-16T10:12:06.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b6dffb640 1 -- 172.21.15.17:0/1098118423 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b680a5ce0 con 0x7f9b700b2ce0 2024-03-16T10:12:06.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 -- 172.21.15.17:0/1098118423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b700b2ce0 msgr2=0x7f9b7014af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 --2- 172.21.15.17:0/1098118423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b700b2ce0 0x7f9b7014af00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9b68009e50 tx=0x7f9b6801c9a0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 -- 172.21.15.17:0/1098118423 shutdown_connections 2024-03-16T10:12:06.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 --2- 172.21.15.17:0/1098118423 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b700b2ce0 0x7f9b7014af00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 --2- 172.21.15.17:0/1098118423 >> v2:172.21.15.174:3300/0 conn(0x7f9b700b2830 0x7f9b7014a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 -- 172.21.15.17:0/1098118423 >> 172.21.15.17:0/1098118423 conn(0x7f9b700b55c0 msgr2=0x7f9b700b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:06.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 -- 172.21.15.17:0/1098118423 shutdown_connections 2024-03-16T10:12:06.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 -- 172.21.15.17:0/1098118423 wait complete. 2024-03-16T10:12:06.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 Processor -- start 2024-03-16T10:12:06.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 -- start start 2024-03-16T10:12:06.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b7006d750 0x7f9b7013ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b700b2830 0x7f9b70071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b700b2ce0 0x7f9b701e02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9b70154870 con 0x7f9b700b2ce0 2024-03-16T10:12:06.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b701546f0 con 0x7f9b7006d750 2024-03-16T10:12:06.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b758d3640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f9b701549f0 con 0x7f9b700b2830 2024-03-16T10:12:06.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b6f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b700b2ce0 0x7f9b701e02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b6f7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b700b2ce0 0x7f9b701e02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55546/0 (socket says 172.21.15.17:55546) 2024-03-16T10:12:06.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b6f7fe640 1 -- 172.21.15.17:0/86652328 learned_addr learned my addr 172.21.15.17:0/86652328 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:06.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b6e7fc640 1 --2- 172.21.15.17:0/86652328 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b700b2830 0x7f9b70071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b6effd640 1 --2- 172.21.15.17:0/86652328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b7006d750 0x7f9b7013ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b6f7fe640 1 -- 172.21.15.17:0/86652328 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b700b2830 msgr2=0x7f9b70071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.819+0000 7f9b6f7fe640 1 --2- 172.21.15.17:0/86652328 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b700b2830 0x7f9b70071dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b6f7fe640 1 -- 172.21.15.17:0/86652328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b7006d750 msgr2=0x7f9b7013ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b6f7fe640 1 --2- 172.21.15.17:0/86652328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b7006d750 0x7f9b7013ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b6f7fe640 1 -- 172.21.15.17:0/86652328 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7013e610 con 0x7f9b700b2ce0 2024-03-16T10:12:06.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b6f7fe640 1 --2- 172.21.15.17:0/86652328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b700b2ce0 0x7f9b701e02c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9b680af040 tx=0x7f9b68019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:06.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b57fff640 1 -- 172.21.15.17:0/86652328 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b680bc020 con 0x7f9b700b2ce0 2024-03-16T10:12:06.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b57fff640 1 -- 172.21.15.17:0/86652328 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b680b8670 con 0x7f9b700b2ce0 2024-03-16T10:12:06.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b758d3640 1 -- 172.21.15.17:0/86652328 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7013f450 con 0x7f9b700b2ce0 2024-03-16T10:12:06.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b57fff640 1 -- 172.21.15.17:0/86652328 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b680b8a60 con 0x7f9b700b2ce0 2024-03-16T10:12:06.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b758d3640 1 -- 172.21.15.17:0/86652328 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b70071200 con 0x7f9b700b2ce0 2024-03-16T10:12:06.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b758d3640 1 -- 172.21.15.17:0/86652328 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b70068360 con 0x7f9b700b2ce0 2024-03-16T10:12:06.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b57fff640 1 -- 172.21.15.17:0/86652328 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f9b680bf020 con 0x7f9b700b2ce0 2024-03-16T10:12:06.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b57fff640 1 --2- 172.21.15.17:0/86652328 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9b5c067660 0x7f9b5c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:06.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b57fff640 1 -- 172.21.15.17:0/86652328 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f9b681206a0 con 0x7f9b700b2ce0 2024-03-16T10:12:06.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b6effd640 1 --2- 172.21.15.17:0/86652328 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9b5c067660 0x7f9b5c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:06.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.823+0000 7f9b6effd640 1 --2- 172.21.15.17:0/86652328 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9b5c067660 0x7f9b5c069b20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9b70144c20 tx=0x7f9b60009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:06.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.827+0000 7f9b57fff640 1 -- 172.21.15.17:0/86652328 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9b680c6030 con 0x7f9b700b2ce0 2024-03-16T10:12:06.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.987+0000 7f9b758d3640 1 -- 172.21.15.17:0/86652328 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f9b7013f620 con 0x7f9b5c067660 2024-03-16T10:12:06.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.987+0000 7f9b57fff640 1 -- 172.21.15.17:0/86652328 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9b7013f620 con 0x7f9b5c067660 2024-03-16T10:12:06.994 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 2 to deep-scrub 2024-03-16T10:12:06.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.991+0000 7f9b758d3640 1 -- 172.21.15.17:0/86652328 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9b5c067660 msgr2=0x7f9b5c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.991+0000 7f9b758d3640 1 --2- 172.21.15.17:0/86652328 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9b5c067660 0x7f9b5c069b20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9b70144c20 tx=0x7f9b60009040 comp rx=0 tx=0).stop 2024-03-16T10:12:06.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.991+0000 7f9b758d3640 1 -- 172.21.15.17:0/86652328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b700b2ce0 msgr2=0x7f9b701e02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:06.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.991+0000 7f9b758d3640 1 --2- 172.21.15.17:0/86652328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b700b2ce0 0x7f9b701e02c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9b680af040 tx=0x7f9b68019740 comp rx=0 tx=0).stop 2024-03-16T10:12:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.991+0000 7f9b758d3640 1 -- 172.21.15.17:0/86652328 shutdown_connections 2024-03-16T10:12:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.991+0000 7f9b758d3640 1 --2- 172.21.15.17:0/86652328 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f9b5c067660 0x7f9b5c069b20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.991+0000 7f9b758d3640 1 --2- 172.21.15.17:0/86652328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9b700b2ce0 0x7f9b701e02c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.991+0000 7f9b758d3640 1 --2- 172.21.15.17:0/86652328 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9b700b2830 0x7f9b70071dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.991+0000 7f9b758d3640 1 --2- 172.21.15.17:0/86652328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b7006d750 0x7f9b7013ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:06.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.991+0000 7f9b758d3640 1 -- 172.21.15.17:0/86652328 >> 172.21.15.17:0/86652328 conn(0x7f9b700b55c0 msgr2=0x7f9b7013d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.991+0000 7f9b758d3640 1 -- 172.21.15.17:0/86652328 shutdown_connections 2024-03-16T10:12:06.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:06.991+0000 7f9b758d3640 1 -- 172.21.15.17:0/86652328 wait complete. 2024-03-16T10:12:07.013 INFO:tasks.ceph:Scrubbing osd.3 2024-03-16T10:12:07.013 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-16T10:12:07.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.135+0000 7fecc455f640 1 Processor -- start 2024-03-16T10:12:07.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.135+0000 7fecc455f640 1 -- start start 2024-03-16T10:12:07.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.135+0000 7fecc455f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecbc0b4200 0x7fecbc150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.135+0000 7fecc455f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecbc0af0c0 con 0x7fecbc0b2400 2024-03-16T10:12:07.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.135+0000 7fecc455f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecbc0b27e0 con 0x7fecbc0ae060 2024-03-16T10:12:07.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.135+0000 7fecc455f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fecbc0afcf0 con 0x7fecbc0b4200 2024-03-16T10:12:07.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.135+0000 7fecc22fb640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fecbc0b2400 0x7fecbc1452c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54118/0 (socket says 172.21.15.17:54118) 2024-03-16T10:12:07.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.135+0000 7fecc22fb640 1 -- 172.21.15.17:0/876440925 learned_addr learned my addr 172.21.15.17:0/876440925 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:12:07.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.135+0000 7fecc2afc640 1 --2- 172.21.15.17:0/876440925 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecbc0b4200 0x7fecbc150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.135+0000 7fecc12f9640 1 -- 172.21.15.17:0/876440925 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 166989475 0 0) 0x7fecbc0af0c0 con 0x7fecbc0b2400 2024-03-16T10:12:07.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.135+0000 7fecc12f9640 1 -- 172.21.15.17:0/876440925 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feca8003530 con 0x7fecbc0b2400 2024-03-16T10:12:07.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.135+0000 7fecc2afc640 1 -- 172.21.15.17:0/876440925 >> v1:172.21.15.174:6789/0 conn(0x7fecbc0ae060 legacy=0x7fecbc072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.135+0000 7fecc2afc640 1 -- 172.21.15.17:0/876440925 >> v1:172.21.15.17:6789/0 conn(0x7fecbc0b2400 legacy=0x7fecbc1452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.135+0000 7fecc2afc640 1 -- 172.21.15.17:0/876440925 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecbc0af980 con 0x7fecbc0b4200 2024-03-16T10:12:07.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc2afc640 1 --2- 172.21.15.17:0/876440925 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecbc0b4200 0x7fecbc150bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fecb4009dc0 tx=0x7fecb401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4b1e214c21edf9c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:07.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc12f9640 1 -- 172.21.15.17:0/876440925 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40b2070 con 0x7fecbc0b4200 2024-03-16T10:12:07.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc12f9640 1 -- 172.21.15.17:0/876440925 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb40a5960 con 0x7fecbc0b4200 2024-03-16T10:12:07.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc12f9640 1 -- 172.21.15.17:0/876440925 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40a5d10 con 0x7fecbc0b4200 2024-03-16T10:12:07.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 -- 172.21.15.17:0/876440925 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecbc0b4200 msgr2=0x7fecbc150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 --2- 172.21.15.17:0/876440925 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecbc0b4200 0x7fecbc150bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fecb4009dc0 tx=0x7fecb401c990 comp rx=0 tx=0).stop 2024-03-16T10:12:07.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 -- 172.21.15.17:0/876440925 shutdown_connections 2024-03-16T10:12:07.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 --2- 172.21.15.17:0/876440925 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecbc0b4200 0x7fecbc150bd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 -- 172.21.15.17:0/876440925 >> 172.21.15.17:0/876440925 conn(0x7fecbc0b8e90 msgr2=0x7fecbc0b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:07.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 -- 172.21.15.17:0/876440925 shutdown_connections 2024-03-16T10:12:07.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 -- 172.21.15.17:0/876440925 wait complete. 2024-03-16T10:12:07.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 Processor -- start 2024-03-16T10:12:07.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 -- start start 2024-03-16T10:12:07.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fecbc0ae060 0x7fecbc1e3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecbc0b2400 0x7fecbc06f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecbc0b4200 0x7fecbc1dfda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc153930 con 0x7fecbc0ae060 2024-03-16T10:12:07.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc0b3fe0 con 0x7fecbc0b4200 2024-03-16T10:12:07.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.139+0000 7fecc455f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fecbc153ab0 con 0x7fecbc0b2400 2024-03-16T10:12:07.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc22fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fecbc0ae060 0x7fecbc1e3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc22fb640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fecbc0ae060 0x7fecbc1e3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55554/0 (socket says 172.21.15.17:55554) 2024-03-16T10:12:07.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc22fb640 1 -- 172.21.15.17:0/1510497084 learned_addr learned my addr 172.21.15.17:0/1510497084 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:07.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc1afa640 1 --2- 172.21.15.17:0/1510497084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecbc0b2400 0x7fecbc06f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc22fb640 1 -- 172.21.15.17:0/1510497084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecbc0b2400 msgr2=0x7fecbc06f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc22fb640 1 --2- 172.21.15.17:0/1510497084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecbc0b2400 0x7fecbc06f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc22fb640 1 -- 172.21.15.17:0/1510497084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecbc0b4200 msgr2=0x7fecbc1dfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc22fb640 1 --2- 172.21.15.17:0/1510497084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecbc0b4200 0x7fecbc1dfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc22fb640 1 -- 172.21.15.17:0/1510497084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecbc071e40 con 0x7fecbc0ae060 2024-03-16T10:12:07.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc1afa640 1 --2- 172.21.15.17:0/1510497084 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fecbc0b2400 0x7fecbc06f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:12:07.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc22fb640 1 --2- 172.21.15.17:0/1510497084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fecbc0ae060 0x7fecbc1e3df0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fecb800c4d0 tx=0x7fecb800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:07.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecb2ffd640 1 -- 172.21.15.17:0/1510497084 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8017070 con 0x7fecbc0ae060 2024-03-16T10:12:07.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecb2ffd640 1 -- 172.21.15.17:0/1510497084 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb8005cf0 con 0x7fecbc0ae060 2024-03-16T10:12:07.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc455f640 1 -- 172.21.15.17:0/1510497084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecbc1e3310 con 0x7fecbc0ae060 2024-03-16T10:12:07.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecb2ffd640 1 -- 172.21.15.17:0/1510497084 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8007270 con 0x7fecbc0ae060 2024-03-16T10:12:07.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc455f640 1 -- 172.21.15.17:0/1510497084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecbc1e38d0 con 0x7fecbc0ae060 2024-03-16T10:12:07.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc455f640 1 -- 172.21.15.17:0/1510497084 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fecbc068360 con 0x7fecbc0ae060 2024-03-16T10:12:07.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecb2ffd640 1 -- 172.21.15.17:0/1510497084 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fecb801a070 con 0x7fecbc0ae060 2024-03-16T10:12:07.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecb2ffd640 1 --2- 172.21.15.17:0/1510497084 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feca0067690 0x7feca0069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecb2ffd640 1 -- 172.21.15.17:0/1510497084 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fecb8084fe0 con 0x7fecbc0ae060 2024-03-16T10:12:07.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecb2ffd640 1 --2- 172.21.15.17:0/1510497084 >> [v2:172.21.15.17:6803/1266087291,v1:172.21.15.17:6806/1266087291] conn(0x7feca006d550 0x7feca006f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecb2ffd640 1 -- 172.21.15.17:0/1510497084 --> [v2:172.21.15.17:6803/1266087291,v1:172.21.15.17:6806/1266087291] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feca0070020 con 0x7feca006d550 2024-03-16T10:12:07.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecb2ffd640 1 -- 172.21.15.17:0/1510497084 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fecb8085470 con 0x7fecbc0ae060 2024-03-16T10:12:07.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc1afa640 1 --2- 172.21.15.17:0/1510497084 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feca0067690 0x7feca0069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc2afc640 1 --2- 172.21.15.17:0/1510497084 >> [v2:172.21.15.17:6803/1266087291,v1:172.21.15.17:6806/1266087291] conn(0x7feca006d550 0x7feca006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc2afc640 1 --2- 172.21.15.17:0/1510497084 >> [v2:172.21.15.17:6803/1266087291,v1:172.21.15.17:6806/1266087291] conn(0x7feca006d550 0x7feca006f970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:07.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.143+0000 7fecc1afa640 1 --2- 172.21.15.17:0/1510497084 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feca0067690 0x7feca0069b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fecbc137870 tx=0x7fecac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:07.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.147+0000 7fecb2ffd640 1 -- 172.21.15.17:0/1510497084 <== osd.3 v2:172.21.15.17:6803/1266087291 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feca0070020 con 0x7feca006d550 2024-03-16T10:12:07.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.159+0000 7fecc455f640 1 -- 172.21.15.17:0/1510497084 --> [v2:172.21.15.17:6803/1266087291,v1:172.21.15.17:6806/1266087291] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fecbc143970 con 0x7feca006d550 2024-03-16T10:12:07.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.159+0000 7fecb2ffd640 1 -- 172.21.15.17:0/1510497084 <== osd.3 v2:172.21.15.17:6803/1266087291 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fecbc143970 con 0x7feca006d550 2024-03-16T10:12:07.166 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-16T10:12:07.166 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T10:12:07.166 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-16T10:12:07.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.159+0000 7fecc455f640 1 -- 172.21.15.17:0/1510497084 >> [v2:172.21.15.17:6803/1266087291,v1:172.21.15.17:6806/1266087291] conn(0x7feca006d550 msgr2=0x7feca006f970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.159+0000 7fecc455f640 1 --2- 172.21.15.17:0/1510497084 >> [v2:172.21.15.17:6803/1266087291,v1:172.21.15.17:6806/1266087291] conn(0x7feca006d550 0x7feca006f970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.159+0000 7fecc455f640 1 -- 172.21.15.17:0/1510497084 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feca0067690 msgr2=0x7feca0069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.159+0000 7fecc455f640 1 --2- 172.21.15.17:0/1510497084 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7feca0067690 0x7feca0069b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fecbc137870 tx=0x7fecac009040 comp rx=0 tx=0).stop 2024-03-16T10:12:07.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.159+0000 7fecc455f640 1 -- 172.21.15.17:0/1510497084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fecbc0ae060 msgr2=0x7fecbc1e3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.159+0000 7fecc455f640 1 --2- 172.21.15.17:0/1510497084 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fecbc0ae060 0x7fecbc1e3df0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fecb800c4d0 tx=0x7fecb800c990 comp rx=0 tx=0).stop 2024-03-16T10:12:07.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.159+0000 7fecc2afc640 1 -- 172.21.15.17:0/1510497084 reap_dead start 2024-03-16T10:12:07.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.159+0000 7fecc455f640 1 -- 172.21.15.17:0/1510497084 shutdown_connections 2024-03-16T10:12:07.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.159+0000 7fecc455f640 1 -- 172.21.15.17:0/1510497084 >> 172.21.15.17:0/1510497084 conn(0x7fecbc0b8e90 msgr2=0x7fecbc0b5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:07.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.159+0000 7fecc455f640 1 -- 172.21.15.17:0/1510497084 shutdown_connections 2024-03-16T10:12:07.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.159+0000 7fecc455f640 1 -- 172.21.15.17:0/1510497084 wait complete. 2024-03-16T10:12:07.182 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-16T10:12:07.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 Processor -- start 2024-03-16T10:12:07.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 -- start start 2024-03-16T10:12:07.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4c0072ab0 0x7fc4c00b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fc4c009fa30 0x7fc4c0147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4c006b8d0 con 0x7fc4c006b4f0 2024-03-16T10:12:07.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c00b4d10 con 0x7fc4c0072ab0 2024-03-16T10:12:07.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fc4c00b52d0 con 0x7fc4c009fa30 2024-03-16T10:12:07.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4bdd9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4c0072ab0 0x7fc4c00b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4bdd9b640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fc4c0072ab0 0x7fc4c00b4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55560/0 (socket says 172.21.15.17:55560) 2024-03-16T10:12:07.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4bdd9b640 1 -- 172.21.15.17:0/3262835993 learned_addr learned my addr 172.21.15.17:0/3262835993 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:07.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4bed9d640 1 --2- 172.21.15.17:0/3262835993 >> v2:172.21.15.174:3300/0 conn(0x7fc4c009fa30 0x7fc4c0147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4bdd9b640 1 -- 172.21.15.17:0/3262835993 >> v2:172.21.15.174:3300/0 conn(0x7fc4c009fa30 msgr2=0x7fc4c0147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:07.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4bdd9b640 1 --2- 172.21.15.17:0/3262835993 >> v2:172.21.15.174:3300/0 conn(0x7fc4c009fa30 0x7fc4c0147bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4bdd9b640 1 -- 172.21.15.17:0/3262835993 >> v1:172.21.15.174:6789/0 conn(0x7fc4c006b4f0 legacy=0x7fc4c009f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:07.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4bdd9b640 1 -- 172.21.15.17:0/3262835993 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c00b4f60 con 0x7fc4c0072ab0 2024-03-16T10:12:07.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4bdd9b640 1 --2- 172.21.15.17:0/3262835993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c0072ab0 0x7fc4c00b4530 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00b4e90 tx=0x7fc4a801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cf1ce893abbaf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:07.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4bd59a640 1 -- 172.21.15.17:0/3262835993 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80b0070 con 0x7fc4c0072ab0 2024-03-16T10:12:07.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4bd59a640 1 -- 172.21.15.17:0/3262835993 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a80ab600 con 0x7fc4c0072ab0 2024-03-16T10:12:07.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4bd59a640 1 -- 172.21.15.17:0/3262835993 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80ab9d0 con 0x7fc4c0072ab0 2024-03-16T10:12:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 -- 172.21.15.17:0/3262835993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c0072ab0 msgr2=0x7fc4c00b4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 --2- 172.21.15.17:0/3262835993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c0072ab0 0x7fc4c00b4530 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00b4e90 tx=0x7fc4a801c3c0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 -- 172.21.15.17:0/3262835993 shutdown_connections 2024-03-16T10:12:07.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 --2- 172.21.15.17:0/3262835993 >> v2:172.21.15.174:3300/0 conn(0x7fc4c009fa30 0x7fc4c0147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 --2- 172.21.15.17:0/3262835993 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c0072ab0 0x7fc4c00b4530 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 -- 172.21.15.17:0/3262835993 >> 172.21.15.17:0/3262835993 conn(0x7fc4c00b9a30 msgr2=0x7fc4c00b9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:07.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 -- 172.21.15.17:0/3262835993 shutdown_connections 2024-03-16T10:12:07.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 -- 172.21.15.17:0/3262835993 wait complete. 2024-03-16T10:12:07.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 Processor -- start 2024-03-16T10:12:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.295+0000 7fc4c4935640 1 -- start start 2024-03-16T10:12:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4c4935640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c006b4f0 0x7fc4c01e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4c4935640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4c0072ab0 0x7fc4c00b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4c4935640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c009fa30 0x7fc4c00b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4c4935640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c0154960 con 0x7fc4c006b4f0 2024-03-16T10:12:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4c4935640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c01547e0 con 0x7fc4c009fa30 2024-03-16T10:12:07.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4c4935640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fc4c0154ae0 con 0x7fc4c0072ab0 2024-03-16T10:12:07.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4be59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c006b4f0 0x7fc4c01e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4be59c640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c006b4f0 0x7fc4c01e34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55564/0 (socket says 172.21.15.17:55564) 2024-03-16T10:12:07.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4be59c640 1 -- 172.21.15.17:0/2422562023 learned_addr learned my addr 172.21.15.17:0/2422562023 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:07.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4be59c640 1 -- 172.21.15.17:0/2422562023 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4c0072ab0 msgr2=0x7fc4c00b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4bdd9b640 1 --2- 172.21.15.17:0/2422562023 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4c0072ab0 0x7fc4c00b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4bed9d640 1 --2- 172.21.15.17:0/2422562023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c009fa30 0x7fc4c00b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4be59c640 1 --2- 172.21.15.17:0/2422562023 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4c0072ab0 0x7fc4c00b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4be59c640 1 -- 172.21.15.17:0/2422562023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c009fa30 msgr2=0x7fc4c00b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4be59c640 1 --2- 172.21.15.17:0/2422562023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c009fa30 0x7fc4c00b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4be59c640 1 -- 172.21.15.17:0/2422562023 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c00b4200 con 0x7fc4c006b4f0 2024-03-16T10:12:07.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4be59c640 1 --2- 172.21.15.17:0/2422562023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c006b4f0 0x7fc4c01e34b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000c7c0 tx=0x7fc4b000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:07.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4b6ffd640 1 -- 172.21.15.17:0/2422562023 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0016020 con 0x7fc4c006b4f0 2024-03-16T10:12:07.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4b6ffd640 1 -- 172.21.15.17:0/2422562023 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b000aa50 con 0x7fc4c006b4f0 2024-03-16T10:12:07.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4c4935640 1 -- 172.21.15.17:0/2422562023 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c01e4100 con 0x7fc4c006b4f0 2024-03-16T10:12:07.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4b6ffd640 1 -- 172.21.15.17:0/2422562023 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0012420 con 0x7fc4c006b4f0 2024-03-16T10:12:07.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4c4935640 1 -- 172.21.15.17:0/2422562023 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c00b3550 con 0x7fc4c006b4f0 2024-03-16T10:12:07.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4c4935640 1 -- 172.21.15.17:0/2422562023 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c01e39f0 con 0x7fc4c006b4f0 2024-03-16T10:12:07.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4b6ffd640 1 -- 172.21.15.17:0/2422562023 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fc4b0019020 con 0x7fc4c006b4f0 2024-03-16T10:12:07.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4b6ffd640 1 --2- 172.21.15.17:0/2422562023 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc4a0067660 0x7fc4a0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4b6ffd640 1 -- 172.21.15.17:0/2422562023 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fc4b00858e0 con 0x7fc4c006b4f0 2024-03-16T10:12:07.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.299+0000 7fc4bdd9b640 1 --2- 172.21.15.17:0/2422562023 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc4a0067660 0x7fc4a0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.303+0000 7fc4bdd9b640 1 --2- 172.21.15.17:0/2422562023 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc4a0067660 0x7fc4a0069b20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80ad000 tx=0x7fc4a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:07.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.303+0000 7fc4b6ffd640 1 -- 172.21.15.17:0/2422562023 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc4b0020020 con 0x7fc4c006b4f0 2024-03-16T10:12:07.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.463+0000 7fc4c4935640 1 -- 172.21.15.17:0/2422562023 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fc4c00b9430 con 0x7fc4a0067660 2024-03-16T10:12:07.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.463+0000 7fc4b6ffd640 1 -- 172.21.15.17:0/2422562023 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc4c00b9430 con 0x7fc4a0067660 2024-03-16T10:12:07.469 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 3 to deep-scrub 2024-03-16T10:12:07.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.463+0000 7fc4c4935640 1 -- 172.21.15.17:0/2422562023 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc4a0067660 msgr2=0x7fc4a0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.463+0000 7fc4c4935640 1 --2- 172.21.15.17:0/2422562023 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc4a0067660 0x7fc4a0069b20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc4a80ad000 tx=0x7fc4a8008040 comp rx=0 tx=0).stop 2024-03-16T10:12:07.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.463+0000 7fc4c4935640 1 -- 172.21.15.17:0/2422562023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c006b4f0 msgr2=0x7fc4c01e34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.463+0000 7fc4c4935640 1 --2- 172.21.15.17:0/2422562023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c006b4f0 0x7fc4c01e34b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc4b000c7c0 tx=0x7fc4b000cc80 comp rx=0 tx=0).stop 2024-03-16T10:12:07.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.463+0000 7fc4c4935640 1 -- 172.21.15.17:0/2422562023 shutdown_connections 2024-03-16T10:12:07.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.463+0000 7fc4c4935640 1 --2- 172.21.15.17:0/2422562023 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fc4a0067660 0x7fc4a0069b20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.463+0000 7fc4c4935640 1 --2- 172.21.15.17:0/2422562023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c009fa30 0x7fc4c00b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.463+0000 7fc4c4935640 1 --2- 172.21.15.17:0/2422562023 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fc4c0072ab0 0x7fc4c00b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.463+0000 7fc4c4935640 1 --2- 172.21.15.17:0/2422562023 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4c006b4f0 0x7fc4c01e34b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.463+0000 7fc4c4935640 1 -- 172.21.15.17:0/2422562023 >> 172.21.15.17:0/2422562023 conn(0x7fc4c00b9a30 msgr2=0x7fc4c0143740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:07.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.463+0000 7fc4c4935640 1 -- 172.21.15.17:0/2422562023 shutdown_connections 2024-03-16T10:12:07.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.463+0000 7fc4c4935640 1 -- 172.21.15.17:0/2422562023 wait complete. 2024-03-16T10:12:07.486 INFO:tasks.ceph:Scrubbing osd.4 2024-03-16T10:12:07.486 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-16T10:12:07.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0c2be9640 1 Processor -- start 2024-03-16T10:12:07.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0c2be9640 1 -- start start 2024-03-16T10:12:07.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0c2be9640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7fb0bc0b0bc0 0x7fb0bc1483b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0c2be9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0bc0b1ac0 con 0x7fb0bc0b4800 2024-03-16T10:12:07.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0c2be9640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0bc0b4be0 con 0x7fb0bc0b2680 2024-03-16T10:12:07.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0c2be9640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7fb0bc0b1220 con 0x7fb0bc0b0bc0 2024-03-16T10:12:07.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0c0985640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fb0bc0b4800 0x7fb0bc0b1f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54128/0 (socket says 172.21.15.17:54128) 2024-03-16T10:12:07.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0c0985640 1 -- 172.21.15.17:0/2053215565 learned_addr learned my addr 172.21.15.17:0/2053215565 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:12:07.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 -- 172.21.15.17:0/2053215565 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 306086566 0 0) 0x7fb0bc0b1ac0 con 0x7fb0bc0b4800 2024-03-16T10:12:07.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 -- 172.21.15.17:0/2053215565 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0a4005740 con 0x7fb0bc0b4800 2024-03-16T10:12:07.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0c1186640 1 --2- 172.21.15.17:0/2053215565 >> v2:172.21.15.174:3300/0 conn(0x7fb0bc0b0bc0 0x7fb0bc1483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 -- 172.21.15.17:0/2053215565 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3761405677 0 0) 0x7fb0a4005740 con 0x7fb0bc0b4800 2024-03-16T10:12:07.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 -- 172.21.15.17:0/2053215565 >> v2:172.21.15.174:3300/0 conn(0x7fb0bc0b0bc0 msgr2=0x7fb0bc1483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:07.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 --2- 172.21.15.17:0/2053215565 >> v2:172.21.15.174:3300/0 conn(0x7fb0bc0b0bc0 0x7fb0bc1483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 -- 172.21.15.17:0/2053215565 >> v1:172.21.15.174:6789/0 conn(0x7fb0bc0b2680 legacy=0x7fb0bc147ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 -- 172.21.15.17:0/2053215565 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc0b1500 con 0x7fb0bc0b4800 2024-03-16T10:12:07.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 -- 172.21.15.17:0/2053215565 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fb0ac001680 con 0x7fb0bc0b4800 2024-03-16T10:12:07.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 -- 172.21.15.17:0/2053215565 >> v1:172.21.15.17:6789/0 conn(0x7fb0bc0b4800 legacy=0x7fb0bc0b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 --2- 172.21.15.17:0/2053215565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a4005bd0 0x7fb0a4005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 --2- 172.21.15.17:0/2053215565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a4006a20 0x7fb0a4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 --2- 172.21.15.17:0/2053215565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0a40073e0 0x7fb0a400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 -- 172.21.15.17:0/2053215565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc0b1220 con 0x7fb0a4006a20 2024-03-16T10:12:07.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 -- 172.21.15.17:0/2053215565 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a400e380 con 0x7fb0a4005bd0 2024-03-16T10:12:07.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bb7fe640 1 -- 172.21.15.17:0/2053215565 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a400e580 con 0x7fb0a40073e0 2024-03-16T10:12:07.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bbfff640 1 --2- 172.21.15.17:0/2053215565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a4006a20 0x7fb0a4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0c0985640 1 --2- 172.21.15.17:0/2053215565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a4005bd0 0x7fb0a4005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0c1186640 1 --2- 172.21.15.17:0/2053215565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0a40073e0 0x7fb0a400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bbfff640 1 -- 172.21.15.17:0/2053215565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0a40073e0 msgr2=0x7fb0a400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:07.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bbfff640 1 --2- 172.21.15.17:0/2053215565 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0a40073e0 0x7fb0a400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bbfff640 1 -- 172.21.15.17:0/2053215565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a4005bd0 msgr2=0x7fb0a4005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:07.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bbfff640 1 --2- 172.21.15.17:0/2053215565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a4005bd0 0x7fb0a4005ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0bbfff640 1 -- 172.21.15.17:0/2053215565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a4005740 con 0x7fb0a4006a20 2024-03-16T10:12:07.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.607+0000 7fb0c1186640 1 -- 172.21.15.17:0/2053215565 reap_dead start 2024-03-16T10:12:07.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0bbfff640 1 --2- 172.21.15.17:0/2053215565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a4006a20 0x7fb0a4006ea0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0001780 tx=0x7fb0b001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a424642a11d055e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:07.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0bb7fe640 1 -- 172.21.15.17:0/2053215565 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00013a0 con 0x7fb0a4006a20 2024-03-16T10:12:07.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0bb7fe640 1 -- 172.21.15.17:0/2053215565 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b00052f0 con 0x7fb0a4006a20 2024-03-16T10:12:07.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0bb7fe640 1 -- 172.21.15.17:0/2053215565 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0003840 con 0x7fb0a4006a20 2024-03-16T10:12:07.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2053215565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a4006a20 msgr2=0x7fb0a4006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 --2- 172.21.15.17:0/2053215565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a4006a20 0x7fb0a4006ea0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0001780 tx=0x7fb0b001c400 comp rx=0 tx=0).stop 2024-03-16T10:12:07.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2053215565 shutdown_connections 2024-03-16T10:12:07.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 --2- 172.21.15.17:0/2053215565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0a4006a20 0x7fb0a4006ea0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2053215565 >> 172.21.15.17:0/2053215565 conn(0x7fb0bc0b8e90 msgr2=0x7fb0bc0b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:07.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2053215565 shutdown_connections 2024-03-16T10:12:07.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2053215565 wait complete. 2024-03-16T10:12:07.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 Processor -- start 2024-03-16T10:12:07.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 -- start start 2024-03-16T10:12:07.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a4006a20 0x7fb0bc1422a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0bc142a30 0x7fb0bc13dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0bc13e4f0 0x7fb0bc13cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc148630 con 0x7fb0bc13e4f0 2024-03-16T10:12:07.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0bc1484b0 con 0x7fb0a4006a20 2024-03-16T10:12:07.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c2be9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fb0bc1487b0 con 0x7fb0bc142a30 2024-03-16T10:12:07.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c1186640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0bc13e4f0 0x7fb0bc13cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c1186640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0bc13e4f0 0x7fb0bc13cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55580/0 (socket says 172.21.15.17:55580) 2024-03-16T10:12:07.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.611+0000 7fb0c1186640 1 -- 172.21.15.17:0/2659540250 learned_addr learned my addr 172.21.15.17:0/2659540250 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:07.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0c0985640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a4006a20 0x7fb0bc1422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0c1186640 1 -- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0bc142a30 msgr2=0x7fb0bc13dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0bbfff640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0bc142a30 0x7fb0bc13dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0c1186640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0bc142a30 0x7fb0bc13dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0c1186640 1 -- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a4006a20 msgr2=0x7fb0bc1422a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0c1186640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a4006a20 0x7fb0bc1422a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0c1186640 1 -- 172.21.15.17:0/2659540250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0bc1dfbb0 con 0x7fb0bc13e4f0 2024-03-16T10:12:07.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0bbfff640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fb0bc142a30 0x7fb0bc13dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:12:07.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0c1186640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0bc13e4f0 0x7fb0bc13cc80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc073b00 tx=0x7fb0b40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:07.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0b97fa640 1 -- 172.21.15.17:0/2659540250 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b4011db0 con 0x7fb0bc13e4f0 2024-03-16T10:12:07.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0b97fa640 1 -- 172.21.15.17:0/2659540250 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0b401b070 con 0x7fb0bc13e4f0 2024-03-16T10:12:07.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2659540250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0bc13fb00 con 0x7fb0bc13e4f0 2024-03-16T10:12:07.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0b97fa640 1 -- 172.21.15.17:0/2659540250 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40039a0 con 0x7fb0bc13e4f0 2024-03-16T10:12:07.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2659540250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0bc09aa40 con 0x7fb0bc13e4f0 2024-03-16T10:12:07.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2659540250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb0bc1dff50 con 0x7fb0bc13e4f0 2024-03-16T10:12:07.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0b97fa640 1 -- 172.21.15.17:0/2659540250 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fb0b4003b40 con 0x7fb0bc13e4f0 2024-03-16T10:12:07.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0b97fa640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 0x7fb0a8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0b97fa640 1 -- 172.21.15.17:0/2659540250 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fb0b4086920 con 0x7fb0bc13e4f0 2024-03-16T10:12:07.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0b97fa640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:6819/326911164,v1:172.21.15.174:6822/326911164] conn(0x7fb0a806d520 0x7fb0a806f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0b97fa640 1 -- 172.21.15.17:0/2659540250 --> [v2:172.21.15.174:6819/326911164,v1:172.21.15.174:6822/326911164] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0b4017e50 con 0x7fb0a806d520 2024-03-16T10:12:07.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0b97fa640 1 -- 172.21.15.17:0/2659540250 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fb0b4086d80 con 0x7fb0bc13e4f0 2024-03-16T10:12:07.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0c0985640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 0x7fb0a8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0bbfff640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:6819/326911164,v1:172.21.15.174:6822/326911164] conn(0x7fb0a806d520 0x7fb0a806f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0c0985640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 0x7fb0a8069b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0017f0 tx=0x7fb0ac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:07.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.615+0000 7fb0bbfff640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:6819/326911164,v1:172.21.15.174:6822/326911164] conn(0x7fb0a806d520 0x7fb0a806f940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:07.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.619+0000 7fb0b97fa640 1 -- 172.21.15.17:0/2659540250 <== osd.4 v2:172.21.15.174:6819/326911164 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0b4017e50 con 0x7fb0a806d520 2024-03-16T10:12:07.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.631+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2659540250 --> [v2:172.21.15.174:6819/326911164,v1:172.21.15.174:6822/326911164] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fb0bc1df9b0 con 0x7fb0a806d520 2024-03-16T10:12:07.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.631+0000 7fb0b97fa640 1 -- 172.21.15.17:0/2659540250 <== osd.4 v2:172.21.15.174:6819/326911164 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fb0bc1df9b0 con 0x7fb0a806d520 2024-03-16T10:12:07.639 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-16T10:12:07.639 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T10:12:07.639 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-16T10:12:07.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.631+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:6819/326911164,v1:172.21.15.174:6822/326911164] conn(0x7fb0a806d520 msgr2=0x7fb0a806f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.631+0000 7fb0c2be9640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:6819/326911164,v1:172.21.15.174:6822/326911164] conn(0x7fb0a806d520 0x7fb0a806f940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.631+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 msgr2=0x7fb0a8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.631+0000 7fb0c2be9640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fb0a8067660 0x7fb0a8069b20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0017f0 tx=0x7fb0ac009040 comp rx=0 tx=0).stop 2024-03-16T10:12:07.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.631+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2659540250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0bc13e4f0 msgr2=0x7fb0bc13cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.631+0000 7fb0c2be9640 1 --2- 172.21.15.17:0/2659540250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb0bc13e4f0 0x7fb0bc13cc80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc073b00 tx=0x7fb0b40115a0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.631+0000 7fb0c1186640 1 -- 172.21.15.17:0/2659540250 reap_dead start 2024-03-16T10:12:07.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.635+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2659540250 shutdown_connections 2024-03-16T10:12:07.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.635+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2659540250 >> 172.21.15.17:0/2659540250 conn(0x7fb0bc0b8e90 msgr2=0x7fb0bc0b8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:07.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.635+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2659540250 shutdown_connections 2024-03-16T10:12:07.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.635+0000 7fb0c2be9640 1 -- 172.21.15.17:0/2659540250 wait complete. 2024-03-16T10:12:07.654 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-16T10:12:07.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f05484a9640 1 Processor -- start 2024-03-16T10:12:07.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f05484a9640 1 -- start start 2024-03-16T10:12:07.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f05484a9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f054006d750 0x7f054006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f05484a9640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f0540145280 0x7f054013f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f05484a9640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f054006db30 con 0x7f05400b2820 2024-03-16T10:12:07.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f05484a9640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f05400b2d30 con 0x7f0540145280 2024-03-16T10:12:07.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f05484a9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f054006fcf0 con 0x7f054006d750 2024-03-16T10:12:07.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f0545a44640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f05400b2820 0x7f054013eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54138/0 (socket says 172.21.15.17:54138) 2024-03-16T10:12:07.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f0545a44640 1 -- 172.21.15.17:0/689215256 learned_addr learned my addr 172.21.15.17:0/689215256 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:12:07.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f0546a46640 1 --2- 172.21.15.17:0/689215256 >> v2:172.21.15.17:3300/0 conn(0x7f0540145280 0x7f054013f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f0546245640 1 --2- 172.21.15.17:0/689215256 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f054006d750 0x7f054006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f0545243640 1 -- 172.21.15.17:0/689215256 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1088479828 0 0) 0x7f054006db30 con 0x7f05400b2820 2024-03-16T10:12:07.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f0545243640 1 -- 172.21.15.17:0/689215256 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f052c003530 con 0x7f05400b2820 2024-03-16T10:12:07.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f0546a46640 1 -- 172.21.15.17:0/689215256 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f054006d750 msgr2=0x7f054006f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:07.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f0546a46640 1 --2- 172.21.15.17:0/689215256 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f054006d750 0x7f054006f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f0546a46640 1 -- 172.21.15.17:0/689215256 >> v1:172.21.15.17:6789/0 conn(0x7f05400b2820 legacy=0x7f054013eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.759+0000 7f0546a46640 1 -- 172.21.15.17:0/689215256 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05400afaf0 con 0x7f0540145280 2024-03-16T10:12:07.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f0546a46640 1 --2- 172.21.15.17:0/689215256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0540145280 0x7f054013f5f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f053c0097f0 tx=0x7f053c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8030f18819c0b859 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:07.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f0545243640 1 -- 172.21.15.17:0/689215256 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c0b0070 con 0x7f0540145280 2024-03-16T10:12:07.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f0545243640 1 -- 172.21.15.17:0/689215256 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f053c0ab600 con 0x7f0540145280 2024-03-16T10:12:07.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f0545243640 1 -- 172.21.15.17:0/689215256 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c0ab9d0 con 0x7f0540145280 2024-03-16T10:12:07.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 -- 172.21.15.17:0/689215256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0540145280 msgr2=0x7f054013f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 --2- 172.21.15.17:0/689215256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0540145280 0x7f054013f5f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f053c0097f0 tx=0x7f053c01c380 comp rx=0 tx=0).stop 2024-03-16T10:12:07.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 -- 172.21.15.17:0/689215256 shutdown_connections 2024-03-16T10:12:07.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 --2- 172.21.15.17:0/689215256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0540145280 0x7f054013f5f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 --2- 172.21.15.17:0/689215256 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f054006d750 0x7f054006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 -- 172.21.15.17:0/689215256 >> 172.21.15.17:0/689215256 conn(0x7f05400b55c0 msgr2=0x7f05400b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:07.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 -- 172.21.15.17:0/689215256 shutdown_connections 2024-03-16T10:12:07.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 -- 172.21.15.17:0/689215256 wait complete. 2024-03-16T10:12:07.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 Processor -- start 2024-03-16T10:12:07.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 -- start start 2024-03-16T10:12:07.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f054006d750 0x7f0540072580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05400b2820 0x7f05401e2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0540145280 0x7f05401e3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0540152550 con 0x7f0540145280 2024-03-16T10:12:07.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f05400b49f0 con 0x7f054006d750 2024-03-16T10:12:07.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f05484a9640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f05401526d0 con 0x7f05400b2820 2024-03-16T10:12:07.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f0546a46640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0540145280 0x7f05401e3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f0546a46640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0540145280 0x7f05401e3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55596/0 (socket says 172.21.15.17:55596) 2024-03-16T10:12:07.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f0546a46640 1 -- 172.21.15.17:0/2705730105 learned_addr learned my addr 172.21.15.17:0/2705730105 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:07.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f0545a44640 1 --2- 172.21.15.17:0/2705730105 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05400b2820 0x7f05401e2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f0546245640 1 --2- 172.21.15.17:0/2705730105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f054006d750 0x7f0540072580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f0546a46640 1 -- 172.21.15.17:0/2705730105 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05400b2820 msgr2=0x7f05401e2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f0546a46640 1 --2- 172.21.15.17:0/2705730105 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05400b2820 0x7f05401e2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f0546a46640 1 -- 172.21.15.17:0/2705730105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f054006d750 msgr2=0x7f0540072580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f0546a46640 1 --2- 172.21.15.17:0/2705730105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f054006d750 0x7f0540072580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.763+0000 7f0546a46640 1 -- 172.21.15.17:0/2705730105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f054014f890 con 0x7f0540145280 2024-03-16T10:12:07.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.767+0000 7f0546a46640 1 --2- 172.21.15.17:0/2705730105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0540145280 0x7f05401e3500 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f053c019fb0 tx=0x7f053c019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:07.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.767+0000 7f0536ffd640 1 -- 172.21.15.17:0/2705730105 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c0ba070 con 0x7f0540145280 2024-03-16T10:12:07.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.767+0000 7f0536ffd640 1 -- 172.21.15.17:0/2705730105 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f053c0b5670 con 0x7f0540145280 2024-03-16T10:12:07.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.767+0000 7f05484a9640 1 -- 172.21.15.17:0/2705730105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05400aefa0 con 0x7f0540145280 2024-03-16T10:12:07.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.767+0000 7f0536ffd640 1 -- 172.21.15.17:0/2705730105 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053c0b0040 con 0x7f0540145280 2024-03-16T10:12:07.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.767+0000 7f05484a9640 1 -- 172.21.15.17:0/2705730105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05400b13a0 con 0x7f0540145280 2024-03-16T10:12:07.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.767+0000 7f05484a9640 1 -- 172.21.15.17:0/2705730105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0540068360 con 0x7f0540145280 2024-03-16T10:12:07.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.767+0000 7f0536ffd640 1 -- 172.21.15.17:0/2705730105 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f053c0bd070 con 0x7f0540145280 2024-03-16T10:12:07.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.767+0000 7f0536ffd640 1 --2- 172.21.15.17:0/2705730105 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0524067660 0x7f0524069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:07.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.767+0000 7f0536ffd640 1 -- 172.21.15.17:0/2705730105 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f053c11fbe0 con 0x7f0540145280 2024-03-16T10:12:07.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.767+0000 7f0546245640 1 --2- 172.21.15.17:0/2705730105 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0524067660 0x7f0524069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:07.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.767+0000 7f0546245640 1 --2- 172.21.15.17:0/2705730105 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0524067660 0x7f0524069b20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0538009e90 tx=0x7f053800b040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:07.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.771+0000 7f0536ffd640 1 -- 172.21.15.17:0/2705730105 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f053c016090 con 0x7f0540145280 2024-03-16T10:12:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.923+0000 7f05484a9640 1 -- 172.21.15.17:0/2705730105 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f05401438b0 con 0x7f0524067660 2024-03-16T10:12:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.927+0000 7f0536ffd640 1 -- 172.21.15.17:0/2705730105 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f05401438b0 con 0x7f0524067660 2024-03-16T10:12:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 4 to deep-scrub 2024-03-16T10:12:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.927+0000 7f05484a9640 1 -- 172.21.15.17:0/2705730105 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0524067660 msgr2=0x7f0524069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.927+0000 7f05484a9640 1 --2- 172.21.15.17:0/2705730105 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0524067660 0x7f0524069b20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0538009e90 tx=0x7f053800b040 comp rx=0 tx=0).stop 2024-03-16T10:12:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.927+0000 7f05484a9640 1 -- 172.21.15.17:0/2705730105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0540145280 msgr2=0x7f05401e3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.927+0000 7f05484a9640 1 --2- 172.21.15.17:0/2705730105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0540145280 0x7f05401e3500 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f053c019fb0 tx=0x7f053c019120 comp rx=0 tx=0).stop 2024-03-16T10:12:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.927+0000 7f05484a9640 1 -- 172.21.15.17:0/2705730105 shutdown_connections 2024-03-16T10:12:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.927+0000 7f05484a9640 1 --2- 172.21.15.17:0/2705730105 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f0524067660 0x7f0524069b20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.927+0000 7f05484a9640 1 --2- 172.21.15.17:0/2705730105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0540145280 0x7f05401e3500 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.927+0000 7f05484a9640 1 --2- 172.21.15.17:0/2705730105 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f05400b2820 0x7f05401e2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.927+0000 7f05484a9640 1 --2- 172.21.15.17:0/2705730105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f054006d750 0x7f0540072580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.927+0000 7f05484a9640 1 -- 172.21.15.17:0/2705730105 >> 172.21.15.17:0/2705730105 conn(0x7f05400b55c0 msgr2=0x7f05400b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.927+0000 7f05484a9640 1 -- 172.21.15.17:0/2705730105 shutdown_connections 2024-03-16T10:12:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:07.927+0000 7f05484a9640 1 -- 172.21.15.17:0/2705730105 wait complete. 2024-03-16T10:12:07.951 INFO:tasks.ceph:Scrubbing osd.5 2024-03-16T10:12:07.951 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-16T10:12:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09cba6a640 1 Processor -- start 2024-03-16T10:12:08.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09cba6a640 1 -- start start 2024-03-16T10:12:08.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09cba6a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09c40b2400 0x7f09c41452c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09cba6a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09c40ae060 0x7f09c4072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09cba6a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09c40af0c0 con 0x7f09c40730c0 2024-03-16T10:12:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09cba6a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f09c40afcf0 con 0x7f09c40b2400 2024-03-16T10:12:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09cba6a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f09c40b3fe0 con 0x7f09c40ae060 2024-03-16T10:12:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09c9806640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09c40b2400 0x7f09c41452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09c9806640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f09c40b2400 0x7f09c41452c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55602/0 (socket says 172.21.15.17:55602) 2024-03-16T10:12:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09c9806640 1 -- 172.21.15.17:0/4223694413 learned_addr learned my addr 172.21.15.17:0/4223694413 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09ca007640 1 --1- 172.21.15.17:0/4223694413 >> v1:172.21.15.17:6789/0 conn(0x7f09c40730c0 0x7f09c406c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54154/0 (socket says 172.21.15.17:54154) 2024-03-16T10:12:08.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09c9005640 1 --2- 172.21.15.17:0/4223694413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09c40ae060 0x7f09c4072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09c9806640 1 -- 172.21.15.17:0/4223694413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09c40ae060 msgr2=0x7f09c4072b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09c9806640 1 --2- 172.21.15.17:0/4223694413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09c40ae060 0x7f09c4072b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09c9806640 1 -- 172.21.15.17:0/4223694413 >> v1:172.21.15.17:6789/0 conn(0x7f09c40730c0 legacy=0x7f09c406c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09c9806640 1 -- 172.21.15.17:0/4223694413 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c40b27e0 con 0x7f09c40b2400 2024-03-16T10:12:08.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.071+0000 7f09c8804640 1 -- 172.21.15.17:0/4223694413 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3176729483 0 0) 0x7f09c40af0c0 con 0x7f09c40730c0 2024-03-16T10:12:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c9806640 1 --2- 172.21.15.17:0/4223694413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09c40b2400 0x7f09c41452c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009870 tx=0x7f09c001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2063634db88c016c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:08.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c8804640 1 -- 172.21.15.17:0/4223694413 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c00a9640 con 0x7f09c40b2400 2024-03-16T10:12:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c8804640 1 -- 172.21.15.17:0/4223694413 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09c00a97e0 con 0x7f09c40b2400 2024-03-16T10:12:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c8804640 1 -- 172.21.15.17:0/4223694413 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c00a9bb0 con 0x7f09c40b2400 2024-03-16T10:12:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 -- 172.21.15.17:0/4223694413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09c40b2400 msgr2=0x7f09c41452c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 --2- 172.21.15.17:0/4223694413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09c40b2400 0x7f09c41452c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009870 tx=0x7f09c001c560 comp rx=0 tx=0).stop 2024-03-16T10:12:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 -- 172.21.15.17:0/4223694413 shutdown_connections 2024-03-16T10:12:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 --2- 172.21.15.17:0/4223694413 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09c40ae060 0x7f09c4072b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 --2- 172.21.15.17:0/4223694413 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09c40b2400 0x7f09c41452c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 -- 172.21.15.17:0/4223694413 >> 172.21.15.17:0/4223694413 conn(0x7f09c40b8e90 msgr2=0x7f09c40b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:08.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 -- 172.21.15.17:0/4223694413 shutdown_connections 2024-03-16T10:12:08.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 -- 172.21.15.17:0/4223694413 wait complete. 2024-03-16T10:12:08.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 Processor -- start 2024-03-16T10:12:08.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 -- start start 2024-03-16T10:12:08.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09c40730c0 0x7f09c4070440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09c40ae060 0x7f09c41de350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c40b2400 0x7f09c41de890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4145980 con 0x7f09c40ae060 2024-03-16T10:12:08.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09c4145800 con 0x7f09c40b2400 2024-03-16T10:12:08.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f09c4152be0 con 0x7f09c40730c0 2024-03-16T10:12:08.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c9005640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09c40ae060 0x7f09c41de350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c9005640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09c40ae060 0x7f09c41de350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55612/0 (socket says 172.21.15.17:55612) 2024-03-16T10:12:08.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c9005640 1 -- 172.21.15.17:0/3170906053 learned_addr learned my addr 172.21.15.17:0/3170906053 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:08.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c9806640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09c40730c0 0x7f09c4070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09ca007640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c40b2400 0x7f09c41de890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c9005640 1 -- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09c40730c0 msgr2=0x7f09c4070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c9005640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09c40730c0 0x7f09c4070440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c9005640 1 -- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c40b2400 msgr2=0x7f09c41de890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c9005640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c40b2400 0x7f09c41de890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c9005640 1 -- 172.21.15.17:0/3170906053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c40720c0 con 0x7f09c40ae060 2024-03-16T10:12:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c9806640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f09c40730c0 0x7f09c4070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:12:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09c9005640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09c40ae060 0x7f09c41de350 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f09b400d800 tx=0x7f09b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09ba7fc640 1 -- 172.21.15.17:0/3170906053 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b400b6c0 con 0x7f09c40ae060 2024-03-16T10:12:08.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09ba7fc640 1 -- 172.21.15.17:0/3170906053 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b4019070 con 0x7f09c40ae060 2024-03-16T10:12:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 -- 172.21.15.17:0/3170906053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c4141ee0 con 0x7f09c40ae060 2024-03-16T10:12:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09ba7fc640 1 -- 172.21.15.17:0/3170906053 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b400bae0 con 0x7f09c40ae060 2024-03-16T10:12:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.075+0000 7f09cba6a640 1 -- 172.21.15.17:0/3170906053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c4070be0 con 0x7f09c40ae060 2024-03-16T10:12:08.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.079+0000 7f09cba6a640 1 -- 172.21.15.17:0/3170906053 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f09c4068360 con 0x7f09c40ae060 2024-03-16T10:12:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.079+0000 7f09ba7fc640 1 -- 172.21.15.17:0/3170906053 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f09b400bc80 con 0x7f09c40ae060 2024-03-16T10:12:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.079+0000 7f09ba7fc640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f09a8067660 0x7f09a8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.079+0000 7f09ba7fc640 1 -- 172.21.15.17:0/3170906053 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f09b4086510 con 0x7f09c40ae060 2024-03-16T10:12:08.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.079+0000 7f09ba7fc640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:6800/1721037852,v1:172.21.15.174:6801/1721037852] conn(0x7f09a806d520 0x7f09a806f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.079+0000 7f09ba7fc640 1 -- 172.21.15.17:0/3170906053 --> [v2:172.21.15.174:6800/1721037852,v1:172.21.15.174:6801/1721037852] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09b4001d40 con 0x7f09a806d520 2024-03-16T10:12:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.079+0000 7f09ba7fc640 1 -- 172.21.15.17:0/3170906053 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f09b40868c0 con 0x7f09c40ae060 2024-03-16T10:12:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.079+0000 7f09c9806640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f09a8067660 0x7f09a8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.079+0000 7f09ca007640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:6800/1721037852,v1:172.21.15.174:6801/1721037852] conn(0x7f09a806d520 0x7f09a806f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.079+0000 7f09c9806640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f09a8067660 0x7f09a8069b20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f09c00ae000 tx=0x7f09c0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.079+0000 7f09ca007640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:6800/1721037852,v1:172.21.15.174:6801/1721037852] conn(0x7f09a806d520 0x7f09a806f940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:08.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.083+0000 7f09ba7fc640 1 -- 172.21.15.17:0/3170906053 <== osd.5 v2:172.21.15.174:6800/1721037852 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09b4001d40 con 0x7f09a806d520 2024-03-16T10:12:08.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.095+0000 7f09cba6a640 1 -- 172.21.15.17:0/3170906053 --> [v2:172.21.15.174:6800/1721037852,v1:172.21.15.174:6801/1721037852] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f09c4071de0 con 0x7f09a806d520 2024-03-16T10:12:08.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.095+0000 7f09ba7fc640 1 -- 172.21.15.17:0/3170906053 <== osd.5 v2:172.21.15.174:6800/1721037852 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f09c4071de0 con 0x7f09a806d520 2024-03-16T10:12:08.103 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-16T10:12:08.103 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T10:12:08.103 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-16T10:12:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.095+0000 7f09cba6a640 1 -- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:6800/1721037852,v1:172.21.15.174:6801/1721037852] conn(0x7f09a806d520 msgr2=0x7f09a806f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.095+0000 7f09cba6a640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:6800/1721037852,v1:172.21.15.174:6801/1721037852] conn(0x7f09a806d520 0x7f09a806f940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.095+0000 7f09cba6a640 1 -- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f09a8067660 msgr2=0x7f09a8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.095+0000 7f09cba6a640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f09a8067660 0x7f09a8069b20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f09c00ae000 tx=0x7f09c0009040 comp rx=0 tx=0).stop 2024-03-16T10:12:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.095+0000 7f09cba6a640 1 -- 172.21.15.17:0/3170906053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09c40ae060 msgr2=0x7f09c41de350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.095+0000 7f09cba6a640 1 --2- 172.21.15.17:0/3170906053 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f09c40ae060 0x7f09c41de350 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f09b400d800 tx=0x7f09b400dcc0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.095+0000 7f09ca007640 1 -- 172.21.15.17:0/3170906053 reap_dead start 2024-03-16T10:12:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.095+0000 7f09cba6a640 1 -- 172.21.15.17:0/3170906053 shutdown_connections 2024-03-16T10:12:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.095+0000 7f09cba6a640 1 -- 172.21.15.17:0/3170906053 >> 172.21.15.17:0/3170906053 conn(0x7f09c40b8e90 msgr2=0x7f09c40b5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.095+0000 7f09cba6a640 1 -- 172.21.15.17:0/3170906053 shutdown_connections 2024-03-16T10:12:08.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.095+0000 7f09cba6a640 1 -- 172.21.15.17:0/3170906053 wait complete. 2024-03-16T10:12:08.119 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-16T10:12:08.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.227+0000 7f703ee8f640 1 Processor -- start 2024-03-16T10:12:08.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.227+0000 7f703ee8f640 1 -- start start 2024-03-16T10:12:08.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f703806fd80 0x7f7038072ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70380b6350 con 0x7f7038072a80 2024-03-16T10:12:08.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70380b6810 con 0x7f70380b5f70 2024-03-16T10:12:08.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f70380b65c0 con 0x7f703806fd80 2024-03-16T10:12:08.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7f7038072a80 0x7f703806f670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:54168/0 (socket says 172.21.15.17:54168) 2024-03-16T10:12:08.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 -- 172.21.15.17:0/1601681184 learned_addr learned my addr 172.21.15.17:0/1601681184 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:12:08.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703d42c640 1 --2- 172.21.15.17:0/1601681184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f703806fd80 0x7f7038072ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 -- 172.21.15.17:0/1601681184 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2609571647 0 0) 0x7f70380b6350 con 0x7f7038072a80 2024-03-16T10:12:08.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 -- 172.21.15.17:0/1601681184 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7020003530 con 0x7f7038072a80 2024-03-16T10:12:08.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 -- 172.21.15.17:0/1601681184 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4204742694 0 0) 0x7f7020003530 con 0x7f7038072a80 2024-03-16T10:12:08.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 -- 172.21.15.17:0/1601681184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f703806fd80 msgr2=0x7f7038072ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:08.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 --2- 172.21.15.17:0/1601681184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f703806fd80 0x7f7038072ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 -- 172.21.15.17:0/1601681184 >> v1:172.21.15.174:6789/0 conn(0x7f70380b5f70 legacy=0x7f7038144660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 -- 172.21.15.17:0/1601681184 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70380b6af0 con 0x7f7038072a80 2024-03-16T10:12:08.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 -- 172.21.15.17:0/1601681184 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7f702c001680 con 0x7f7038072a80 2024-03-16T10:12:08.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 -- 172.21.15.17:0/1601681184 >> v1:172.21.15.17:6789/0 conn(0x7f7038072a80 legacy=0x7f703806f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 --2- 172.21.15.17:0/1601681184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f703806fd80 0x7f70200039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 --2- 172.21.15.17:0/1601681184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7020003f70 0x7f7020004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 --2- 172.21.15.17:0/1601681184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7020004d50 0x7f70200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 -- 172.21.15.17:0/1601681184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f70380b65c0 con 0x7f7020003f70 2024-03-16T10:12:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 -- 172.21.15.17:0/1601681184 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f702000bc10 con 0x7f7020004d50 2024-03-16T10:12:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 -- 172.21.15.17:0/1601681184 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f702000be10 con 0x7f703806fd80 2024-03-16T10:12:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 --2- 172.21.15.17:0/1601681184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7020003f70 0x7f7020004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703cc2b640 1 --2- 172.21.15.17:0/1601681184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f703806fd80 0x7f70200039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703d42c640 1 --2- 172.21.15.17:0/1601681184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7020004d50 0x7f70200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 -- 172.21.15.17:0/1601681184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f703806fd80 msgr2=0x7f70200039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 --2- 172.21.15.17:0/1601681184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f703806fd80 0x7f70200039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 -- 172.21.15.17:0/1601681184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7020004d50 msgr2=0x7f70200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:08.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 --2- 172.21.15.17:0/1601681184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7020004d50 0x7f70200051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 -- 172.21.15.17:0/1601681184 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7020003530 con 0x7f7020003f70 2024-03-16T10:12:08.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703cc2b640 1 --2- 172.21.15.17:0/1601681184 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f703806fd80 0x7f70200039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:12:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 --2- 172.21.15.17:0/1601681184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7020003f70 0x7f7020004810 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f702c00c1c0 tx=0x7f702c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861155f8200316c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 -- 172.21.15.17:0/1601681184 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c007d00 con 0x7f7020003f70 2024-03-16T10:12:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 -- 172.21.15.17:0/1601681184 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702c0a9020 con 0x7f7020003f70 2024-03-16T10:12:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70377fe640 1 -- 172.21.15.17:0/1601681184 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c005ae0 con 0x7f7020003f70 2024-03-16T10:12:08.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 -- 172.21.15.17:0/1601681184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7020003f70 msgr2=0x7f7020004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 --2- 172.21.15.17:0/1601681184 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7020003f70 0x7f7020004810 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f702c00c1c0 tx=0x7f702c01cc60 comp rx=0 tx=0).stop 2024-03-16T10:12:08.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703d42c640 1 -- 172.21.15.17:0/1601681184 reap_dead start 2024-03-16T10:12:08.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 -- 172.21.15.17:0/1601681184 shutdown_connections 2024-03-16T10:12:08.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 -- 172.21.15.17:0/1601681184 >> 172.21.15.17:0/1601681184 conn(0x7f70380b79d0 msgr2=0x7f70380b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:08.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 -- 172.21.15.17:0/1601681184 shutdown_connections 2024-03-16T10:12:08.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 -- 172.21.15.17:0/1601681184 wait complete. 2024-03-16T10:12:08.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 Processor -- start 2024-03-16T10:12:08.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 -- start start 2024-03-16T10:12:08.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7020003f70 0x7f7038146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70381507c0 0x7f7038140130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7038145030 0x7f703813c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7038153880 con 0x7f70381507c0 2024-03-16T10:12:08.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7038153700 con 0x7f7020003f70 2024-03-16T10:12:08.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f7038153a00 con 0x7f7038145030 2024-03-16T10:12:08.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70381507c0 0x7f7038140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70381507c0 0x7f7038140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55644/0 (socket says 172.21.15.17:55644) 2024-03-16T10:12:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 -- 172.21.15.17:0/3867199389 learned_addr learned my addr 172.21.15.17:0/3867199389 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703d42c640 1 --2- 172.21.15.17:0/3867199389 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7038145030 0x7f703813c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703cc2b640 1 --2- 172.21.15.17:0/3867199389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7020003f70 0x7f7038146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 -- 172.21.15.17:0/3867199389 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7038145030 msgr2=0x7f703813c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 --2- 172.21.15.17:0/3867199389 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7038145030 0x7f703813c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 -- 172.21.15.17:0/3867199389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7020003f70 msgr2=0x7f7038146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 --2- 172.21.15.17:0/3867199389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7020003f70 0x7f7038146600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 -- 172.21.15.17:0/3867199389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7038140cd0 con 0x7f70381507c0 2024-03-16T10:12:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f7037fff640 1 --2- 172.21.15.17:0/3867199389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70381507c0 0x7f7038140130 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f702c0ab040 tx=0x7f7020005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:08.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70357fa640 1 -- 172.21.15.17:0/3867199389 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c0049f0 con 0x7f70381507c0 2024-03-16T10:12:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70357fa640 1 -- 172.21.15.17:0/3867199389 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f702c004b90 con 0x7f70381507c0 2024-03-16T10:12:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 -- 172.21.15.17:0/3867199389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7038145d90 con 0x7f70381507c0 2024-03-16T10:12:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f70357fa640 1 -- 172.21.15.17:0/3867199389 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c0070d0 con 0x7f70381507c0 2024-03-16T10:12:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.235+0000 7f703ee8f640 1 -- 172.21.15.17:0/3867199389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7038147710 con 0x7f70381507c0 2024-03-16T10:12:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.239+0000 7f703ee8f640 1 -- 172.21.15.17:0/3867199389 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f703806a130 con 0x7f70381507c0 2024-03-16T10:12:08.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.239+0000 7f70357fa640 1 -- 172.21.15.17:0/3867199389 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f702c007270 con 0x7f70381507c0 2024-03-16T10:12:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.239+0000 7f70357fa640 1 --2- 172.21.15.17:0/3867199389 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7024067660 0x7f7024069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.239+0000 7f70357fa640 1 -- 172.21.15.17:0/3867199389 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f702c11df40 con 0x7f70381507c0 2024-03-16T10:12:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.239+0000 7f703cc2b640 1 --2- 172.21.15.17:0/3867199389 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7024067660 0x7f7024069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.239+0000 7f703cc2b640 1 --2- 172.21.15.17:0/3867199389 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7024067660 0x7f7024069b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7028000c10 tx=0x7f7028009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:08.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.243+0000 7f70357fa640 1 -- 172.21.15.17:0/3867199389 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f702c016970 con 0x7f70381507c0 2024-03-16T10:12:08.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.399+0000 7f703ee8f640 1 -- 172.21.15.17:0/3867199389 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f70380b6350 con 0x7f7024067660 2024-03-16T10:12:08.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.399+0000 7f70357fa640 1 -- 172.21.15.17:0/3867199389 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f70380b6350 con 0x7f7024067660 2024-03-16T10:12:08.406 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 5 to deep-scrub 2024-03-16T10:12:08.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.399+0000 7f703ee8f640 1 -- 172.21.15.17:0/3867199389 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7024067660 msgr2=0x7f7024069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.399+0000 7f703ee8f640 1 --2- 172.21.15.17:0/3867199389 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7024067660 0x7f7024069b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7028000c10 tx=0x7f7028009040 comp rx=0 tx=0).stop 2024-03-16T10:12:08.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.399+0000 7f703ee8f640 1 -- 172.21.15.17:0/3867199389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70381507c0 msgr2=0x7f7038140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.399+0000 7f703ee8f640 1 --2- 172.21.15.17:0/3867199389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70381507c0 0x7f7038140130 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f702c0ab040 tx=0x7f7020005840 comp rx=0 tx=0).stop 2024-03-16T10:12:08.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.403+0000 7f703ee8f640 1 -- 172.21.15.17:0/3867199389 shutdown_connections 2024-03-16T10:12:08.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.403+0000 7f703ee8f640 1 --2- 172.21.15.17:0/3867199389 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f7024067660 0x7f7024069b20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.403+0000 7f703ee8f640 1 --2- 172.21.15.17:0/3867199389 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f7038145030 0x7f703813c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.403+0000 7f703ee8f640 1 --2- 172.21.15.17:0/3867199389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f70381507c0 0x7f7038140130 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.403+0000 7f703ee8f640 1 --2- 172.21.15.17:0/3867199389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7020003f70 0x7f7038146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.403+0000 7f703ee8f640 1 -- 172.21.15.17:0/3867199389 >> 172.21.15.17:0/3867199389 conn(0x7f70380b79d0 msgr2=0x7f703813e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.403+0000 7f703ee8f640 1 -- 172.21.15.17:0/3867199389 shutdown_connections 2024-03-16T10:12:08.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.403+0000 7f703ee8f640 1 -- 172.21.15.17:0/3867199389 wait complete. 2024-03-16T10:12:08.423 INFO:tasks.ceph:Scrubbing osd.6 2024-03-16T10:12:08.423 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-16T10:12:08.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 Processor -- start 2024-03-16T10:12:08.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 -- start start 2024-03-16T10:12:08.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c0b4800 0x7f8c7c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c7c147ca0 0x7f8c7c148130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c7c0b1ac0 con 0x7f8c7c0b1f70 2024-03-16T10:12:08.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f8c7c0b4be0 con 0x7f8c7c147ca0 2024-03-16T10:12:08.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8c7c0b1220 con 0x7f8c7c0b4800 2024-03-16T10:12:08.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c7bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c7c147ca0 0x7f8c7c148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c7bfff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f8c7c147ca0 0x7f8c7c148130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55648/0 (socket says 172.21.15.17:55648) 2024-03-16T10:12:08.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c7bfff640 1 -- 172.21.15.17:0/1399964504 learned_addr learned my addr 172.21.15.17:0/1399964504 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:08.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c7b7fe640 1 --2- 172.21.15.17:0/1399964504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c0b4800 0x7f8c7c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c7bfff640 1 -- 172.21.15.17:0/1399964504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c0b4800 msgr2=0x7f8c7c0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:08.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c7bfff640 1 --2- 172.21.15.17:0/1399964504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c0b4800 0x7f8c7c0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c7bfff640 1 -- 172.21.15.17:0/1399964504 >> v1:172.21.15.174:6789/0 conn(0x7f8c7c0b1f70 legacy=0x7f8c7c0b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c7bfff640 1 -- 172.21.15.17:0/1399964504 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c0b1500 con 0x7f8c7c147ca0 2024-03-16T10:12:08.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c7b7fe640 1 --2- 172.21.15.17:0/1399964504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c0b4800 0x7f8c7c0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T10:12:08.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c7bfff640 1 --2- 172.21.15.17:0/1399964504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c7c147ca0 0x7f8c7c148130 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009e50 tx=0x7f8c7401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288ee2e65769f562 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:08.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c7a7fc640 1 -- 172.21.15.17:0/1399964504 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c740a5770 con 0x7f8c7c147ca0 2024-03-16T10:12:08.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c7a7fc640 1 -- 172.21.15.17:0/1399964504 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c740a5910 con 0x7f8c7c147ca0 2024-03-16T10:12:08.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c7a7fc640 1 -- 172.21.15.17:0/1399964504 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c740a5ce0 con 0x7f8c7c147ca0 2024-03-16T10:12:08.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 -- 172.21.15.17:0/1399964504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c7c147ca0 msgr2=0x7f8c7c148130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 --2- 172.21.15.17:0/1399964504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c7c147ca0 0x7f8c7c148130 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009e50 tx=0x7f8c7401c9a0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 -- 172.21.15.17:0/1399964504 shutdown_connections 2024-03-16T10:12:08.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 --2- 172.21.15.17:0/1399964504 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c7c147ca0 0x7f8c7c148130 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 --2- 172.21.15.17:0/1399964504 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c0b4800 0x7f8c7c0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 -- 172.21.15.17:0/1399964504 >> 172.21.15.17:0/1399964504 conn(0x7f8c7c0b8e90 msgr2=0x7f8c7c0b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:08.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 -- 172.21.15.17:0/1399964504 shutdown_connections 2024-03-16T10:12:08.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.547+0000 7f8c81bd7640 1 -- 172.21.15.17:0/1399964504 wait complete. 2024-03-16T10:12:08.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c81bd7640 1 Processor -- start 2024-03-16T10:12:08.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c81bd7640 1 -- start start 2024-03-16T10:12:08.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c81bd7640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c7c0b1f70 0x7f8c7c1d68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c81bd7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c7c0b4800 0x7f8c7c13ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c81bd7640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c147ca0 0x7f8c7c141330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c81bd7640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c1515b0 con 0x7f8c7c0b1f70 2024-03-16T10:12:08.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c81bd7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7c151430 con 0x7f8c7c0b4800 2024-03-16T10:12:08.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c81bd7640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8c7c151730 con 0x7f8c7c147ca0 2024-03-16T10:12:08.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c7b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c7c0b1f70 0x7f8c7c1d68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c7b7fe640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c7c0b1f70 0x7f8c7c1d68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55656/0 (socket says 172.21.15.17:55656) 2024-03-16T10:12:08.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c7b7fe640 1 -- 172.21.15.17:0/3806307514 learned_addr learned my addr 172.21.15.17:0/3806307514 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:08.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c7bfff640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c147ca0 0x7f8c7c141330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c7affd640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c7c0b4800 0x7f8c7c13ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c7b7fe640 1 -- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c147ca0 msgr2=0x7f8c7c141330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c7b7fe640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c147ca0 0x7f8c7c141330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c7b7fe640 1 -- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c7c0b4800 msgr2=0x7f8c7c13ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c7b7fe640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c7c0b4800 0x7f8c7c13ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c7b7fe640 1 -- 172.21.15.17:0/3806307514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7c13cbe0 con 0x7f8c7c0b1f70 2024-03-16T10:12:08.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c7bfff640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8c7c147ca0 0x7f8c7c141330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:12:08.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c7b7fe640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c7c0b1f70 0x7f8c7c1d68b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00d830 tx=0x7f8c6c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:08.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c63fff640 1 -- 172.21.15.17:0/3806307514 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c00b6c0 con 0x7f8c7c0b1f70 2024-03-16T10:12:08.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c63fff640 1 -- 172.21.15.17:0/3806307514 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c019070 con 0x7f8c7c0b1f70 2024-03-16T10:12:08.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c81bd7640 1 -- 172.21.15.17:0/3806307514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7c13c2b0 con 0x7f8c7c0b1f70 2024-03-16T10:12:08.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c63fff640 1 -- 172.21.15.17:0/3806307514 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c00ba60 con 0x7f8c7c0b1f70 2024-03-16T10:12:08.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c81bd7640 1 -- 172.21.15.17:0/3806307514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c7c13bfc0 con 0x7f8c7c0b1f70 2024-03-16T10:12:08.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c81bd7640 1 -- 172.21.15.17:0/3806307514 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8c7c1d7260 con 0x7f8c7c0b1f70 2024-03-16T10:12:08.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c63fff640 1 -- 172.21.15.17:0/3806307514 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8c6c01e460 con 0x7f8c7c0b1f70 2024-03-16T10:12:08.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c63fff640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8c68067660 0x7f8c68069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c63fff640 1 -- 172.21.15.17:0/3806307514 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f8c6c0865a0 con 0x7f8c7c0b1f70 2024-03-16T10:12:08.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c63fff640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:6803/4101740318,v1:172.21.15.174:6805/4101740318] conn(0x7f8c6806d520 0x7f8c6806f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c63fff640 1 -- 172.21.15.17:0/3806307514 --> [v2:172.21.15.174:6803/4101740318,v1:172.21.15.174:6805/4101740318] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8c6c006820 con 0x7f8c6806d520 2024-03-16T10:12:08.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c63fff640 1 -- 172.21.15.17:0/3806307514 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f8c6c086a30 con 0x7f8c7c0b1f70 2024-03-16T10:12:08.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.551+0000 7f8c7affd640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8c68067660 0x7f8c68069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.555+0000 7f8c7bfff640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:6803/4101740318,v1:172.21.15.174:6805/4101740318] conn(0x7f8c6806d520 0x7f8c6806f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.555+0000 7f8c7affd640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8c68067660 0x7f8c68069b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c140670 tx=0x7f8c70009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:08.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.555+0000 7f8c7bfff640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:6803/4101740318,v1:172.21.15.174:6805/4101740318] conn(0x7f8c6806d520 0x7f8c6806f940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:08.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.555+0000 7f8c63fff640 1 -- 172.21.15.17:0/3806307514 <== osd.6 v2:172.21.15.174:6803/4101740318 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8c6c006820 con 0x7f8c6806d520 2024-03-16T10:12:08.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.571+0000 7f8c81bd7640 1 -- 172.21.15.17:0/3806307514 --> [v2:172.21.15.174:6803/4101740318,v1:172.21.15.174:6805/4101740318] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8c7c068360 con 0x7f8c6806d520 2024-03-16T10:12:08.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.571+0000 7f8c63fff640 1 -- 172.21.15.17:0/3806307514 <== osd.6 v2:172.21.15.174:6803/4101740318 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8c7c068360 con 0x7f8c6806d520 2024-03-16T10:12:08.576 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-16T10:12:08.576 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T10:12:08.576 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-16T10:12:08.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.571+0000 7f8c81bd7640 1 -- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:6803/4101740318,v1:172.21.15.174:6805/4101740318] conn(0x7f8c6806d520 msgr2=0x7f8c6806f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.571+0000 7f8c81bd7640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:6803/4101740318,v1:172.21.15.174:6805/4101740318] conn(0x7f8c6806d520 0x7f8c6806f940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.571+0000 7f8c81bd7640 1 -- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8c68067660 msgr2=0x7f8c68069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.571+0000 7f8c81bd7640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8c68067660 0x7f8c68069b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c140670 tx=0x7f8c70009040 comp rx=0 tx=0).stop 2024-03-16T10:12:08.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.571+0000 7f8c81bd7640 1 -- 172.21.15.17:0/3806307514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c7c0b1f70 msgr2=0x7f8c7c1d68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.571+0000 7f8c81bd7640 1 --2- 172.21.15.17:0/3806307514 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8c7c0b1f70 0x7f8c7c1d68b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00d830 tx=0x7f8c6c00dcf0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.571+0000 7f8c7bfff640 1 -- 172.21.15.17:0/3806307514 reap_dead start 2024-03-16T10:12:08.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.571+0000 7f8c81bd7640 1 -- 172.21.15.17:0/3806307514 shutdown_connections 2024-03-16T10:12:08.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.571+0000 7f8c81bd7640 1 -- 172.21.15.17:0/3806307514 >> 172.21.15.17:0/3806307514 conn(0x7f8c7c0b8e90 msgr2=0x7f8c7c0b9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:08.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.571+0000 7f8c81bd7640 1 -- 172.21.15.17:0/3806307514 shutdown_connections 2024-03-16T10:12:08.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.571+0000 7f8c81bd7640 1 -- 172.21.15.17:0/3806307514 wait complete. 2024-03-16T10:12:08.592 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-16T10:12:08.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd341f640 1 Processor -- start 2024-03-16T10:12:08.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd341f640 1 -- start start 2024-03-16T10:12:08.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd341f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bcc06d750 0x7f8bcc06f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd341f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8bcc145280 0x7f8bcc13f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd341f640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bcc06db30 con 0x7f8bcc0b2820 2024-03-16T10:12:08.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd341f640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f8bcc0b2d30 con 0x7f8bcc145280 2024-03-16T10:12:08.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd341f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc06fcf0 con 0x7f8bcc06d750 2024-03-16T10:12:08.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd11bb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bcc06d750 0x7f8bcc06f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd19bc640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f8bcc145280 0x7f8bcc13f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd11bb640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bcc06d750 0x7f8bcc06f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.17:49308/0 (socket says 172.21.15.17:49308) 2024-03-16T10:12:08.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd11bb640 1 -- 172.21.15.17:0/3172782444 learned_addr learned my addr 172.21.15.17:0/3172782444 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd09ba640 1 --1- >> v1:172.21.15.174:6789/0 conn(0x7f8bcc0b2820 0x7f8bcc13eee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.174:6789/0 says I am v1:172.21.15.17:49128/0 (socket says 172.21.15.17:49128) 2024-03-16T10:12:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd11bb640 1 -- 172.21.15.17:0/3172782444 >> v2:172.21.15.174:3300/0 conn(0x7f8bcc145280 msgr2=0x7f8bcc13f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd11bb640 1 --2- 172.21.15.17:0/3172782444 >> v2:172.21.15.174:3300/0 conn(0x7f8bcc145280 0x7f8bcc13f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd11bb640 1 -- 172.21.15.17:0/3172782444 >> v1:172.21.15.174:6789/0 conn(0x7f8bcc0b2820 legacy=0x7f8bcc13eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd11bb640 1 -- 172.21.15.17:0/3172782444 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc0afaf0 con 0x7f8bcc06d750 2024-03-16T10:12:08.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bd11bb640 1 --2- 172.21.15.17:0/3172782444 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bcc06d750 0x7f8bcc06f7b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009e80 tx=0x7f8bc401c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4af7b67eedff1135 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:08.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.711+0000 7f8bc3fff640 1 -- 172.21.15.17:0/3172782444 <== mon.2 v2:172.21.15.174:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40a55b0 con 0x7f8bcc06d750 2024-03-16T10:12:08.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bc3fff640 1 -- 172.21.15.17:0/3172782444 <== mon.2 v2:172.21.15.174:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc40a5750 con 0x7f8bcc06d750 2024-03-16T10:12:08.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bc3fff640 1 -- 172.21.15.17:0/3172782444 <== mon.2 v2:172.21.15.174:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc40a5b20 con 0x7f8bcc06d750 2024-03-16T10:12:08.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 -- 172.21.15.17:0/3172782444 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bcc06d750 msgr2=0x7f8bcc06f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 --2- 172.21.15.17:0/3172782444 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bcc06d750 0x7f8bcc06f7b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4009e80 tx=0x7f8bc401c8f0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 -- 172.21.15.17:0/3172782444 shutdown_connections 2024-03-16T10:12:08.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 --2- 172.21.15.17:0/3172782444 >> v2:172.21.15.174:3300/0 conn(0x7f8bcc145280 0x7f8bcc13f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 --2- 172.21.15.17:0/3172782444 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bcc06d750 0x7f8bcc06f7b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 -- 172.21.15.17:0/3172782444 >> 172.21.15.17:0/3172782444 conn(0x7f8bcc0b55c0 msgr2=0x7f8bcc0b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:08.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 -- 172.21.15.17:0/3172782444 shutdown_connections 2024-03-16T10:12:08.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 -- 172.21.15.17:0/3172782444 wait complete. 2024-03-16T10:12:08.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 Processor -- start 2024-03-16T10:12:08.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 -- start start 2024-03-16T10:12:08.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bcc06d750 0x7f8bcc13db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bcc0b2820 0x7f8bcc13e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bcc145280 0x7f8bcc13b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc152550 con 0x7f8bcc145280 2024-03-16T10:12:08.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8bcc0b49f0 con 0x7f8bcc06d750 2024-03-16T10:12:08.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f8bcc1526d0 con 0x7f8bcc0b2820 2024-03-16T10:12:08.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd19bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bcc145280 0x7f8bcc13b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd19bc640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bcc145280 0x7f8bcc13b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55664/0 (socket says 172.21.15.17:55664) 2024-03-16T10:12:08.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd19bc640 1 -- 172.21.15.17:0/2882254467 learned_addr learned my addr 172.21.15.17:0/2882254467 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:08.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd09ba640 1 --2- 172.21.15.17:0/2882254467 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bcc0b2820 0x7f8bcc13e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd11bb640 1 --2- 172.21.15.17:0/2882254467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bcc06d750 0x7f8bcc13db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd19bc640 1 -- 172.21.15.17:0/2882254467 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bcc0b2820 msgr2=0x7f8bcc13e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd19bc640 1 --2- 172.21.15.17:0/2882254467 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bcc0b2820 0x7f8bcc13e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd19bc640 1 -- 172.21.15.17:0/2882254467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bcc06d750 msgr2=0x7f8bcc13db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd19bc640 1 --2- 172.21.15.17:0/2882254467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bcc06d750 0x7f8bcc13db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd19bc640 1 -- 172.21.15.17:0/2882254467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bcc13d3c0 con 0x7f8bcc145280 2024-03-16T10:12:08.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd19bc640 1 --2- 172.21.15.17:0/2882254467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bcc145280 0x7f8bcc13b7d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800e7f0 tx=0x7f8bc800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:08.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bc1ffb640 1 -- 172.21.15.17:0/2882254467 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8017070 con 0x7f8bcc145280 2024-03-16T10:12:08.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bc1ffb640 1 -- 172.21.15.17:0/2882254467 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bc800c940 con 0x7f8bcc145280 2024-03-16T10:12:08.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 -- 172.21.15.17:0/2882254467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bcc13d0e0 con 0x7f8bcc145280 2024-03-16T10:12:08.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bc1ffb640 1 -- 172.21.15.17:0/2882254467 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800eea0 con 0x7f8bcc145280 2024-03-16T10:12:08.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.715+0000 7f8bd341f640 1 -- 172.21.15.17:0/2882254467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bcc13a620 con 0x7f8bcc145280 2024-03-16T10:12:08.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.719+0000 7f8bd341f640 1 -- 172.21.15.17:0/2882254467 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bcc068360 con 0x7f8bcc145280 2024-03-16T10:12:08.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.719+0000 7f8bc1ffb640 1 -- 172.21.15.17:0/2882254467 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8bc801a070 con 0x7f8bcc145280 2024-03-16T10:12:08.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.719+0000 7f8bc1ffb640 1 --2- 172.21.15.17:0/2882254467 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8bb0067660 0x7f8bb0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:08.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.719+0000 7f8bc1ffb640 1 -- 172.21.15.17:0/2882254467 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f8bc8085750 con 0x7f8bcc145280 2024-03-16T10:12:08.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.719+0000 7f8bd11bb640 1 --2- 172.21.15.17:0/2882254467 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8bb0067660 0x7f8bb0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:08.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.719+0000 7f8bd11bb640 1 --2- 172.21.15.17:0/2882254467 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8bb0067660 0x7f8bb0069b20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40ae000 tx=0x7f8bc40a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:08.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.723+0000 7f8bc1ffb640 1 -- 172.21.15.17:0/2882254467 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8bc8056760 con 0x7f8bcc145280 2024-03-16T10:12:08.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.883+0000 7f8bd341f640 1 -- 172.21.15.17:0/2882254467 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f8bcc13b4f0 con 0x7f8bb0067660 2024-03-16T10:12:08.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.883+0000 7f8bc1ffb640 1 -- 172.21.15.17:0/2882254467 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8bcc13b4f0 con 0x7f8bb0067660 2024-03-16T10:12:08.891 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 6 to deep-scrub 2024-03-16T10:12:08.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.887+0000 7f8bd341f640 1 -- 172.21.15.17:0/2882254467 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8bb0067660 msgr2=0x7f8bb0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.887+0000 7f8bd341f640 1 --2- 172.21.15.17:0/2882254467 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8bb0067660 0x7f8bb0069b20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40ae000 tx=0x7f8bc40a7070 comp rx=0 tx=0).stop 2024-03-16T10:12:08.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.887+0000 7f8bd341f640 1 -- 172.21.15.17:0/2882254467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bcc145280 msgr2=0x7f8bcc13b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:08.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.887+0000 7f8bd341f640 1 --2- 172.21.15.17:0/2882254467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bcc145280 0x7f8bcc13b7d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800e7f0 tx=0x7f8bc800ecb0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.887+0000 7f8bd341f640 1 -- 172.21.15.17:0/2882254467 shutdown_connections 2024-03-16T10:12:08.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.887+0000 7f8bd341f640 1 --2- 172.21.15.17:0/2882254467 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f8bb0067660 0x7f8bb0069b20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.887+0000 7f8bd341f640 1 --2- 172.21.15.17:0/2882254467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8bcc145280 0x7f8bcc13b7d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.887+0000 7f8bd341f640 1 --2- 172.21.15.17:0/2882254467 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f8bcc0b2820 0x7f8bcc13e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.887+0000 7f8bd341f640 1 --2- 172.21.15.17:0/2882254467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bcc06d750 0x7f8bcc13db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:08.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.887+0000 7f8bd341f640 1 -- 172.21.15.17:0/2882254467 >> 172.21.15.17:0/2882254467 conn(0x7f8bcc0b55c0 msgr2=0x7f8bcc136970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:08.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.887+0000 7f8bd341f640 1 -- 172.21.15.17:0/2882254467 shutdown_connections 2024-03-16T10:12:08.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:08.887+0000 7f8bd341f640 1 -- 172.21.15.17:0/2882254467 wait complete. 2024-03-16T10:12:08.910 INFO:tasks.ceph:Scrubbing osd.7 2024-03-16T10:12:08.911 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-16T10:12:09.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 Processor -- start 2024-03-16T10:12:09.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 -- start start 2024-03-16T10:12:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fd8300b4800 0x7fd8300b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8300b1f70 0x7fd8300b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8300b1ac0 con 0x7fd830147ca0 2024-03-16T10:12:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fd830148840 con 0x7fd8300b4800 2024-03-16T10:12:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd8300b4be0 con 0x7fd8300b1f70 2024-03-16T10:12:09.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd8354a3640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fd830147ca0 0x7fd830148130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:43672/0 (socket says 172.21.15.17:43672) 2024-03-16T10:12:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd8354a3640 1 -- 172.21.15.17:0/987478579 learned_addr learned my addr 172.21.15.17:0/987478579 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:12:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd834ca2640 1 --2- 172.21.15.17:0/987478579 >> v2:172.21.15.17:3300/0 conn(0x7fd8300b4800 0x7fd8300b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd82ffff640 1 --2- 172.21.15.17:0/987478579 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8300b1f70 0x7fd8300b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd82f7fe640 1 -- 172.21.15.17:0/987478579 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3823727248 0 0) 0x7fd8300b1ac0 con 0x7fd830147ca0 2024-03-16T10:12:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd82f7fe640 1 -- 172.21.15.17:0/987478579 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd81c003530 con 0x7fd830147ca0 2024-03-16T10:12:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd834ca2640 1 -- 172.21.15.17:0/987478579 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8300b1f70 msgr2=0x7fd8300b23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd834ca2640 1 --2- 172.21.15.17:0/987478579 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8300b1f70 0x7fd8300b23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd834ca2640 1 -- 172.21.15.17:0/987478579 >> v1:172.21.15.17:6789/0 conn(0x7fd830147ca0 legacy=0x7fd830148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd834ca2640 1 -- 172.21.15.17:0/987478579 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8300b28f0 con 0x7fd8300b4800 2024-03-16T10:12:09.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd834ca2640 1 --2- 172.21.15.17:0/987478579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8300b4800 0x7fd8300b0bc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd8200097c0 tx=0x7fd82001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846aa8715b08d708 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:09.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd82f7fe640 1 -- 172.21.15.17:0/987478579 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8200b0070 con 0x7fd8300b4800 2024-03-16T10:12:09.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd82f7fe640 1 -- 172.21.15.17:0/987478579 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8200ab600 con 0x7fd8300b4800 2024-03-16T10:12:09.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd82f7fe640 1 -- 172.21.15.17:0/987478579 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8200ab9d0 con 0x7fd8300b4800 2024-03-16T10:12:09.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 -- 172.21.15.17:0/987478579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8300b4800 msgr2=0x7fd8300b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 --2- 172.21.15.17:0/987478579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8300b4800 0x7fd8300b0bc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd8200097c0 tx=0x7fd82001c410 comp rx=0 tx=0).stop 2024-03-16T10:12:09.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 -- 172.21.15.17:0/987478579 shutdown_connections 2024-03-16T10:12:09.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 --2- 172.21.15.17:0/987478579 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8300b1f70 0x7fd8300b23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 --2- 172.21.15.17:0/987478579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8300b4800 0x7fd8300b0bc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 -- 172.21.15.17:0/987478579 >> 172.21.15.17:0/987478579 conn(0x7fd8300b8e90 msgr2=0x7fd8300b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:09.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 -- 172.21.15.17:0/987478579 shutdown_connections 2024-03-16T10:12:09.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 -- 172.21.15.17:0/987478579 wait complete. 2024-03-16T10:12:09.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 Processor -- start 2024-03-16T10:12:09.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.039+0000 7fd836f06640 1 -- start start 2024-03-16T10:12:09.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd836f06640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8300b1f70 0x7fd8301429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd836f06640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8300b4800 0x7fd830141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd836f06640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd830147ca0 0x7fd8301e3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd836f06640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd830151430 con 0x7fd8300b1f70 2024-03-16T10:12:09.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd836f06640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8300732e0 con 0x7fd830147ca0 2024-03-16T10:12:09.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd836f06640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fd8301515b0 con 0x7fd8300b4800 2024-03-16T10:12:09.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd834ca2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8300b1f70 0x7fd8301429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd834ca2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8300b1f70 0x7fd8301429c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40266/0 (socket says 172.21.15.17:40266) 2024-03-16T10:12:09.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd834ca2640 1 -- 172.21.15.17:0/2197132160 learned_addr learned my addr 172.21.15.17:0/2197132160 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:09.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd8354a3640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd830147ca0 0x7fd8301e3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd82ffff640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8300b4800 0x7fd830141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd834ca2640 1 -- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8300b4800 msgr2=0x7fd830141ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd834ca2640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8300b4800 0x7fd830141ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd834ca2640 1 -- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd830147ca0 msgr2=0x7fd8301e3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd834ca2640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd830147ca0 0x7fd8301e3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd834ca2640 1 -- 172.21.15.17:0/2197132160 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd830070e50 con 0x7fd8300b1f70 2024-03-16T10:12:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd8354a3640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd830147ca0 0x7fd8301e3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:12:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd82ffff640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fd8300b4800 0x7fd830141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T10:12:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd834ca2640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8300b1f70 0x7fd8301429c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd82001a040 tx=0x7fd82001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:09.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd82d7fa640 1 -- 172.21.15.17:0/2197132160 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8200b0050 con 0x7fd8300b1f70 2024-03-16T10:12:09.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd82d7fa640 1 -- 172.21.15.17:0/2197132160 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8200bc070 con 0x7fd8300b1f70 2024-03-16T10:12:09.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd82d7fa640 1 -- 172.21.15.17:0/2197132160 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd820006e20 con 0x7fd8300b1f70 2024-03-16T10:12:09.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd836f06640 1 -- 172.21.15.17:0/2197132160 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8301430a0 con 0x7fd8300b1f70 2024-03-16T10:12:09.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd836f06640 1 -- 172.21.15.17:0/2197132160 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd83014e840 con 0x7fd8300b1f70 2024-03-16T10:12:09.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd836f06640 1 -- 172.21.15.17:0/2197132160 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd830068360 con 0x7fd8300b1f70 2024-03-16T10:12:09.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd82d7fa640 1 -- 172.21.15.17:0/2197132160 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd8200b5830 con 0x7fd8300b1f70 2024-03-16T10:12:09.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd82d7fa640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd810067660 0x7fd810069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd82d7fa640 1 -- 172.21.15.17:0/2197132160 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fd820120600 con 0x7fd8300b1f70 2024-03-16T10:12:09.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.043+0000 7fd82d7fa640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:6816/1083843796,v1:172.21.15.174:6817/1083843796] conn(0x7fd81006d520 0x7fd81006f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.047+0000 7fd82d7fa640 1 -- 172.21.15.17:0/2197132160 --> [v2:172.21.15.174:6816/1083843796,v1:172.21.15.174:6817/1083843796] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8200ab600 con 0x7fd81006d520 2024-03-16T10:12:09.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.047+0000 7fd82d7fa640 1 -- 172.21.15.17:0/2197132160 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd8201209b0 con 0x7fd8300b1f70 2024-03-16T10:12:09.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.047+0000 7fd82ffff640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd810067660 0x7fd810069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.047+0000 7fd8354a3640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:6816/1083843796,v1:172.21.15.174:6817/1083843796] conn(0x7fd81006d520 0x7fd81006f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.047+0000 7fd82ffff640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd810067660 0x7fd810069b20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd8301dd760 tx=0x7fd818008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:09.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.047+0000 7fd8354a3640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:6816/1083843796,v1:172.21.15.174:6817/1083843796] conn(0x7fd81006d520 0x7fd81006f940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:09.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.047+0000 7fd82d7fa640 1 -- 172.21.15.17:0/2197132160 <== osd.7 v2:172.21.15.174:6816/1083843796 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd8200ab600 con 0x7fd81006d520 2024-03-16T10:12:09.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.063+0000 7fd836f06640 1 -- 172.21.15.17:0/2197132160 --> [v2:172.21.15.174:6816/1083843796,v1:172.21.15.174:6817/1083843796] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd830141980 con 0x7fd81006d520 2024-03-16T10:12:09.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.063+0000 7fd82d7fa640 1 -- 172.21.15.17:0/2197132160 <== osd.7 v2:172.21.15.174:6816/1083843796 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd830141980 con 0x7fd81006d520 2024-03-16T10:12:09.070 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-03-16T10:12:09.070 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T10:12:09.070 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-03-16T10:12:09.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.063+0000 7fd836f06640 1 -- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:6816/1083843796,v1:172.21.15.174:6817/1083843796] conn(0x7fd81006d520 msgr2=0x7fd81006f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.063+0000 7fd836f06640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:6816/1083843796,v1:172.21.15.174:6817/1083843796] conn(0x7fd81006d520 0x7fd81006f940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.063+0000 7fd836f06640 1 -- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd810067660 msgr2=0x7fd810069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.063+0000 7fd836f06640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fd810067660 0x7fd810069b20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd8301dd760 tx=0x7fd818008040 comp rx=0 tx=0).stop 2024-03-16T10:12:09.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.063+0000 7fd836f06640 1 -- 172.21.15.17:0/2197132160 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8300b1f70 msgr2=0x7fd8301429c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.063+0000 7fd836f06640 1 --2- 172.21.15.17:0/2197132160 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd8300b1f70 0x7fd8301429c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd82001a040 tx=0x7fd82001ac50 comp rx=0 tx=0).stop 2024-03-16T10:12:09.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.063+0000 7fd8354a3640 1 -- 172.21.15.17:0/2197132160 reap_dead start 2024-03-16T10:12:09.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.063+0000 7fd836f06640 1 -- 172.21.15.17:0/2197132160 shutdown_connections 2024-03-16T10:12:09.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.063+0000 7fd836f06640 1 -- 172.21.15.17:0/2197132160 >> 172.21.15.17:0/2197132160 conn(0x7fd8300b8e90 msgr2=0x7fd8300b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:09.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.063+0000 7fd836f06640 1 -- 172.21.15.17:0/2197132160 shutdown_connections 2024-03-16T10:12:09.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.063+0000 7fd836f06640 1 -- 172.21.15.17:0/2197132160 wait complete. 2024-03-16T10:12:09.085 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-16T10:12:09.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa77520a640 1 Processor -- start 2024-03-16T10:12:09.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa77520a640 1 -- start start 2024-03-16T10:12:09.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa77520a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa77006d750 0x7fa77006f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa77520a640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa770145280 0x7fa77013f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa77520a640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa77006db30 con 0x7fa7700b2820 2024-03-16T10:12:09.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa77520a640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7fa7700b2d30 con 0x7fa770145280 2024-03-16T10:12:09.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa77520a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa77006fcf0 con 0x7fa77006d750 2024-03-16T10:12:09.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa76f59e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa770145280 0x7fa77013f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa76f59e640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7fa770145280 0x7fa77013f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40274/0 (socket says 172.21.15.17:40274) 2024-03-16T10:12:09.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa76f59e640 1 -- 172.21.15.17:0/1579980654 learned_addr learned my addr 172.21.15.17:0/1579980654 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:09.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa76ed9d640 1 --2- 172.21.15.17:0/1579980654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa77006d750 0x7fa77006f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa76f59e640 1 -- 172.21.15.17:0/1579980654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa77006d750 msgr2=0x7fa77006f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:09.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa76f59e640 1 --2- 172.21.15.17:0/1579980654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa77006d750 0x7fa77006f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa76f59e640 1 -- 172.21.15.17:0/1579980654 >> v1:172.21.15.17:6789/0 conn(0x7fa7700b2820 legacy=0x7fa77013eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa76f59e640 1 -- 172.21.15.17:0/1579980654 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7700afaf0 con 0x7fa770145280 2024-03-16T10:12:09.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.191+0000 7fa76dd9b640 1 -- 172.21.15.17:0/1579980654 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 52864935 0 0) 0x7fa77006db30 con 0x7fa7700b2820 2024-03-16T10:12:09.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.195+0000 7fa76f59e640 1 --2- 172.21.15.17:0/1579980654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa770145280 0x7fa77013f5f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa768009e80 tx=0x7fa76801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea777873692fae87 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:09.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.195+0000 7fa76dd9b640 1 -- 172.21.15.17:0/1579980654 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680a55b0 con 0x7fa770145280 2024-03-16T10:12:09.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.195+0000 7fa76dd9b640 1 -- 172.21.15.17:0/1579980654 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7680a5750 con 0x7fa770145280 2024-03-16T10:12:09.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.195+0000 7fa76dd9b640 1 -- 172.21.15.17:0/1579980654 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680a5b20 con 0x7fa770145280 2024-03-16T10:12:09.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.195+0000 7fa77520a640 1 -- 172.21.15.17:0/1579980654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa770145280 msgr2=0x7fa77013f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.195+0000 7fa77520a640 1 --2- 172.21.15.17:0/1579980654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa770145280 0x7fa77013f5f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa768009e80 tx=0x7fa76801c8f0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.195+0000 7fa77520a640 1 -- 172.21.15.17:0/1579980654 shutdown_connections 2024-03-16T10:12:09.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.195+0000 7fa77520a640 1 --2- 172.21.15.17:0/1579980654 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa770145280 0x7fa77013f5f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.195+0000 7fa77520a640 1 --2- 172.21.15.17:0/1579980654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa77006d750 0x7fa77006f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.195+0000 7fa77520a640 1 -- 172.21.15.17:0/1579980654 >> 172.21.15.17:0/1579980654 conn(0x7fa7700b55c0 msgr2=0x7fa7700b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:09.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.195+0000 7fa77520a640 1 -- 172.21.15.17:0/1579980654 shutdown_connections 2024-03-16T10:12:09.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.195+0000 7fa77520a640 1 -- 172.21.15.17:0/1579980654 wait complete. 2024-03-16T10:12:09.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.195+0000 7fa77520a640 1 Processor -- start 2024-03-16T10:12:09.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.195+0000 7fa77520a640 1 -- start start 2024-03-16T10:12:09.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa77520a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa77006d750 0x7fa7701e2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa77520a640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7700b2820 0x7fa7701e3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa77520a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa770145280 0x7fa770135dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa77520a640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa770152550 con 0x7fa770145280 2024-03-16T10:12:09.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa77520a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7700b49f0 con 0x7fa77006d750 2024-03-16T10:12:09.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa77520a640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fa7701526d0 con 0x7fa7700b2820 2024-03-16T10:12:09.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa76f59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa770145280 0x7fa770135dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa76f59e640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa770145280 0x7fa770135dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40276/0 (socket says 172.21.15.17:40276) 2024-03-16T10:12:09.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa76f59e640 1 -- 172.21.15.17:0/3471146768 learned_addr learned my addr 172.21.15.17:0/3471146768 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:09.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa76ed9d640 1 --2- 172.21.15.17:0/3471146768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa77006d750 0x7fa7701e2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa76e59c640 1 --2- 172.21.15.17:0/3471146768 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7700b2820 0x7fa7701e3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa76f59e640 1 -- 172.21.15.17:0/3471146768 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7700b2820 msgr2=0x7fa7701e3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa76f59e640 1 --2- 172.21.15.17:0/3471146768 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7700b2820 0x7fa7701e3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa76f59e640 1 -- 172.21.15.17:0/3471146768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa77006d750 msgr2=0x7fa7701e2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa76f59e640 1 --2- 172.21.15.17:0/3471146768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa77006d750 0x7fa7701e2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa76f59e640 1 -- 172.21.15.17:0/3471146768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa770137d00 con 0x7fa770145280 2024-03-16T10:12:09.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa76f59e640 1 --2- 172.21.15.17:0/3471146768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa770145280 0x7fa770135dd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa7680af040 tx=0x7fa768019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:09.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa7577fe640 1 -- 172.21.15.17:0/3471146768 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680bc070 con 0x7fa770145280 2024-03-16T10:12:09.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa7577fe640 1 -- 172.21.15.17:0/3471146768 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7680b8850 con 0x7fa770145280 2024-03-16T10:12:09.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa77520a640 1 -- 172.21.15.17:0/3471146768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7701381b0 con 0x7fa770145280 2024-03-16T10:12:09.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa7577fe640 1 -- 172.21.15.17:0/3471146768 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7680a55b0 con 0x7fa770145280 2024-03-16T10:12:09.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa77520a640 1 -- 172.21.15.17:0/3471146768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa770139330 con 0x7fa770145280 2024-03-16T10:12:09.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa77520a640 1 -- 172.21.15.17:0/3471146768 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa770068360 con 0x7fa770145280 2024-03-16T10:12:09.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa7577fe640 1 -- 172.21.15.17:0/3471146768 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa7680bf070 con 0x7fa770145280 2024-03-16T10:12:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa7577fe640 1 --2- 172.21.15.17:0/3471146768 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa75c067660 0x7fa75c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa7577fe640 1 -- 172.21.15.17:0/3471146768 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fa7681203d0 con 0x7fa770145280 2024-03-16T10:12:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.199+0000 7fa76ed9d640 1 --2- 172.21.15.17:0/3471146768 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa75c067660 0x7fa75c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.203+0000 7fa76ed9d640 1 --2- 172.21.15.17:0/3471146768 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa75c067660 0x7fa75c069b20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa7700b08e0 tx=0x7fa760008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:09.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.203+0000 7fa7577fe640 1 -- 172.21.15.17:0/3471146768 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa768016600 con 0x7fa770145280 2024-03-16T10:12:09.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.363+0000 7fa77520a640 1 -- 172.21.15.17:0/3471146768 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fa7700b3a80 con 0x7fa75c067660 2024-03-16T10:12:09.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.367+0000 7fa7577fe640 1 -- 172.21.15.17:0/3471146768 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa7700b3a80 con 0x7fa75c067660 2024-03-16T10:12:09.372 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 7 to deep-scrub 2024-03-16T10:12:09.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.367+0000 7fa77520a640 1 -- 172.21.15.17:0/3471146768 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa75c067660 msgr2=0x7fa75c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.367+0000 7fa77520a640 1 --2- 172.21.15.17:0/3471146768 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa75c067660 0x7fa75c069b20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa7700b08e0 tx=0x7fa760008040 comp rx=0 tx=0).stop 2024-03-16T10:12:09.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.367+0000 7fa77520a640 1 -- 172.21.15.17:0/3471146768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa770145280 msgr2=0x7fa770135dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.367+0000 7fa77520a640 1 --2- 172.21.15.17:0/3471146768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa770145280 0x7fa770135dd0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa7680af040 tx=0x7fa768019690 comp rx=0 tx=0).stop 2024-03-16T10:12:09.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.367+0000 7fa77520a640 1 -- 172.21.15.17:0/3471146768 shutdown_connections 2024-03-16T10:12:09.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.367+0000 7fa77520a640 1 --2- 172.21.15.17:0/3471146768 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fa75c067660 0x7fa75c069b20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.367+0000 7fa77520a640 1 --2- 172.21.15.17:0/3471146768 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa770145280 0x7fa770135dd0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.367+0000 7fa77520a640 1 --2- 172.21.15.17:0/3471146768 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fa7700b2820 0x7fa7701e3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.367+0000 7fa77520a640 1 --2- 172.21.15.17:0/3471146768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa77006d750 0x7fa7701e2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.367+0000 7fa77520a640 1 -- 172.21.15.17:0/3471146768 >> 172.21.15.17:0/3471146768 conn(0x7fa7700b55c0 msgr2=0x7fa770143a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:09.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.367+0000 7fa77520a640 1 -- 172.21.15.17:0/3471146768 shutdown_connections 2024-03-16T10:12:09.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.367+0000 7fa77520a640 1 -- 172.21.15.17:0/3471146768 wait complete. 2024-03-16T10:12:09.390 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:12:09.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.499+0000 7f280db2f640 1 Processor -- start 2024-03-16T10:12:09.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.499+0000 7f280db2f640 1 -- start start 2024-03-16T10:12:09.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2808072ce0 0x7f2808145f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f28080b58d0 0x7f280806ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f280806f220 0x7f2808142540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f28080b5d30 con 0x7f280806f220 2024-03-16T10:12:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f28081454c0 con 0x7f28080b58d0 2024-03-16T10:12:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f28080730c0 con 0x7f2808072ce0 2024-03-16T10:12:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2807fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f280806f220 0x7f2808142540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2807fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f280806f220 0x7f2808142540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40286/0 (socket says 172.21.15.17:40286) 2024-03-16T10:12:09.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2807fff640 1 -- 172.21.15.17:0/2934841101 learned_addr learned my addr 172.21.15.17:0/2934841101 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f28077fe640 1 --2- 172.21.15.17:0/2934841101 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2808072ce0 0x7f2808145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2806ffd640 1 --2- 172.21.15.17:0/2934841101 >> v2:172.21.15.174:3300/0 conn(0x7f28080b58d0 0x7f280806ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2807fff640 1 -- 172.21.15.17:0/2934841101 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2808072ce0 msgr2=0x7f2808145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2807fff640 1 --2- 172.21.15.17:0/2934841101 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2808072ce0 0x7f2808145f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2807fff640 1 -- 172.21.15.17:0/2934841101 >> v2:172.21.15.174:3300/0 conn(0x7f28080b58d0 msgr2=0x7f280806ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2807fff640 1 --2- 172.21.15.17:0/2934841101 >> v2:172.21.15.174:3300/0 conn(0x7f28080b58d0 0x7f280806ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2807fff640 1 -- 172.21.15.17:0/2934841101 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2808145730 con 0x7f280806f220 2024-03-16T10:12:09.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2807fff640 1 --2- 172.21.15.17:0/2934841101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f280806f220 0x7f2808142540 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f280806c1d0 tx=0x7f280001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5a4ab89a57bc45 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f28067fc640 1 -- 172.21.15.17:0/2934841101 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28000a56c0 con 0x7f280806f220 2024-03-16T10:12:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f28067fc640 1 -- 172.21.15.17:0/2934841101 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28000a5860 con 0x7f280806f220 2024-03-16T10:12:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f28067fc640 1 -- 172.21.15.17:0/2934841101 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28000a5c30 con 0x7f280806f220 2024-03-16T10:12:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- 172.21.15.17:0/2934841101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f280806f220 msgr2=0x7f2808142540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 --2- 172.21.15.17:0/2934841101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f280806f220 0x7f2808142540 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f280806c1d0 tx=0x7f280001c930 comp rx=0 tx=0).stop 2024-03-16T10:12:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- 172.21.15.17:0/2934841101 shutdown_connections 2024-03-16T10:12:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 --2- 172.21.15.17:0/2934841101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f280806f220 0x7f2808142540 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 --2- 172.21.15.17:0/2934841101 >> v2:172.21.15.174:3300/0 conn(0x7f28080b58d0 0x7f280806ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 --2- 172.21.15.17:0/2934841101 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f2808072ce0 0x7f2808145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- 172.21.15.17:0/2934841101 >> 172.21.15.17:0/2934841101 conn(0x7f28080b79d0 msgr2=0x7f28080b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- 172.21.15.17:0/2934841101 shutdown_connections 2024-03-16T10:12:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- 172.21.15.17:0/2934841101 wait complete. 2024-03-16T10:12:09.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 Processor -- start 2024-03-16T10:12:09.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- start start 2024-03-16T10:12:09.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f280806f220 0x7f28080b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2808072ce0 0x7f28081e1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28080b58d0 0x7f28081e16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f28081494c0 con 0x7f2808072ce0 2024-03-16T10:12:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2808149340 con 0x7f280806f220 2024-03-16T10:12:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2808149640 con 0x7f28080b58d0 2024-03-16T10:12:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2806ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2808072ce0 0x7f28081e1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2806ffd640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2808072ce0 0x7f28081e1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40302/0 (socket says 172.21.15.17:40302) 2024-03-16T10:12:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2806ffd640 1 -- 172.21.15.17:0/2434938385 learned_addr learned my addr 172.21.15.17:0/2434938385 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:09.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f28077fe640 1 --2- 172.21.15.17:0/2434938385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f280806f220 0x7f28080b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2807fff640 1 --2- 172.21.15.17:0/2434938385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28080b58d0 0x7f28081e16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2806ffd640 1 -- 172.21.15.17:0/2434938385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28080b58d0 msgr2=0x7f28081e16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2806ffd640 1 --2- 172.21.15.17:0/2434938385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28080b58d0 0x7f28081e16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2806ffd640 1 -- 172.21.15.17:0/2434938385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f280806f220 msgr2=0x7f28080b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2806ffd640 1 --2- 172.21.15.17:0/2434938385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f280806f220 0x7f28080b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2806ffd640 1 -- 172.21.15.17:0/2434938385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280809bf60 con 0x7f2808072ce0 2024-03-16T10:12:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f2806ffd640 1 --2- 172.21.15.17:0/2434938385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2808072ce0 0x7f28081e1170 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f27f800d830 tx=0x7f27f800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:09.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f27fffff640 1 -- 172.21.15.17:0/2434938385 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f800b6c0 con 0x7f2808072ce0 2024-03-16T10:12:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f27fffff640 1 -- 172.21.15.17:0/2434938385 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27f8019070 con 0x7f2808072ce0 2024-03-16T10:12:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- 172.21.15.17:0/2434938385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28080b5000 con 0x7f2808072ce0 2024-03-16T10:12:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f27fffff640 1 -- 172.21.15.17:0/2434938385 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f800ba60 con 0x7f2808072ce0 2024-03-16T10:12:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- 172.21.15.17:0/2434938385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28080b1b80 con 0x7f2808072ce0 2024-03-16T10:12:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f280db2f640 1 -- 172.21.15.17:0/2434938385 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280806a130 con 0x7f2808072ce0 2024-03-16T10:12:09.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.503+0000 7f27fffff640 1 -- 172.21.15.17:0/2434938385 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f27f801e460 con 0x7f2808072ce0 2024-03-16T10:12:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.507+0000 7f27fffff640 1 --2- 172.21.15.17:0/2434938385 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f27e8067660 0x7f27e8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.507+0000 7f27fffff640 1 -- 172.21.15.17:0/2434938385 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f27f8085b70 con 0x7f2808072ce0 2024-03-16T10:12:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.507+0000 7f28077fe640 1 --2- 172.21.15.17:0/2434938385 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f27e8067660 0x7f27e8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:09.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.507+0000 7f28077fe640 1 --2- 172.21.15.17:0/2434938385 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f27e8067660 0x7f27e8069b20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2808143b50 tx=0x7f27f0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:09.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.515+0000 7f27fffff640 1 -- 172.21.15.17:0/2434938385 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f280809bf60 con 0x7f2808072ce0 2024-03-16T10:12:09.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.659+0000 7f280db2f640 1 -- 172.21.15.17:0/2434938385 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f28080b2030 con 0x7f27e8067660 2024-03-16T10:12:09.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.667+0000 7f27fffff640 1 -- 172.21.15.17:0/2434938385 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13479 (secure 0 0 0) 0x7f28080b2030 con 0x7f27e8067660 2024-03-16T10:12:09.669 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:12:09.669 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-16T10:12:09.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.667+0000 7f280db2f640 1 -- 172.21.15.17:0/2434938385 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f27e8067660 msgr2=0x7f27e8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.667+0000 7f280db2f640 1 --2- 172.21.15.17:0/2434938385 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f27e8067660 0x7f27e8069b20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2808143b50 tx=0x7f27f0008040 comp rx=0 tx=0).stop 2024-03-16T10:12:09.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.667+0000 7f280db2f640 1 -- 172.21.15.17:0/2434938385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2808072ce0 msgr2=0x7f28081e1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:09.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.667+0000 7f280db2f640 1 --2- 172.21.15.17:0/2434938385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2808072ce0 0x7f28081e1170 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f27f800d830 tx=0x7f27f800dcf0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.667+0000 7f280db2f640 1 -- 172.21.15.17:0/2434938385 shutdown_connections 2024-03-16T10:12:09.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.667+0000 7f280db2f640 1 --2- 172.21.15.17:0/2434938385 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f27e8067660 0x7f27e8069b20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.667+0000 7f280db2f640 1 --2- 172.21.15.17:0/2434938385 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f28080b58d0 0x7f28081e16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.667+0000 7f280db2f640 1 --2- 172.21.15.17:0/2434938385 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2808072ce0 0x7f28081e1170 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.667+0000 7f280db2f640 1 --2- 172.21.15.17:0/2434938385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f280806f220 0x7f28080b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:09.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.667+0000 7f280db2f640 1 -- 172.21.15.17:0/2434938385 >> 172.21.15.17:0/2434938385 conn(0x7f28080b79d0 msgr2=0x7f28080741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:09.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.667+0000 7f280db2f640 1 -- 172.21.15.17:0/2434938385 shutdown_connections 2024-03-16T10:12:09.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:09.667+0000 7f280db2f640 1 -- 172.21.15.17:0/2434938385 wait complete. 2024-03-16T10:12:09.686 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-03-16T10:12:08.038127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236944,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737776,"statfs":{"total":773094113280,"available":772851482624,"internally_reserved":0,"allocated":4235264,"data_stored":2351392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-55,"num_write_kb":-27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002250"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-16T10:12:01.675362+0000","last_change":"2024-03-16T10:10:48.248686+0000","last_active":"2024-03-16T10:12:01.675362+0000","last_peered":"2024-03-16T10:12:01.675362+0000","last_clean":"2024-03-16T10:12:01.675362+0000","last_became_active":"2024-03-16T10:10:48.243054+0000","last_became_peered":"2024-03-16T10:10:48.243054+0000","last_unstale":"2024-03-16T10:12:01.675362+0000","last_undegraded":"2024-03-16T10:12:01.675362+0000","last_fullsized":"2024-03-16T10:12:01.675362+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T10:10:46.517938+0000","last_clean_scrub_stamp":"2024-03-16T10:10:46.517938+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:33:42.906473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":974848,"data_stored":736700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":736700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:12:09.688 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-16T10:10:46.517938+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=16, tm_hour=10, tm_min=10, tm_sec=46, tm_wday=5, tm_yday=76, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=16, tm_hour=10, tm_min=12, tm_sec=4, tm_wday=5, tm_yday=76, tm_isdst=0) 2024-03-16T10:12:09.689 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-16T10:12:29.689 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T10:12:29.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.819+0000 7fbfff4a2640 1 Processor -- start 2024-03-16T10:12:29.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfff4a2640 1 -- start start 2024-03-16T10:12:29.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfff4a2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbff8145d70 0x7fbff8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:29.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfff4a2640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbff806db30 con 0x7fbff806d750 2024-03-16T10:12:29.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfff4a2640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbff80b4c30 con 0x7fbff80affc0 2024-03-16T10:12:29.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfff4a2640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fbff8073380 con 0x7fbff8145d70 2024-03-16T10:12:29.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbffd23e640 1 --1- >> v1:172.21.15.17:6789/0 conn(0x7fbff806d750 0x7fbff80b0c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.17:6789/0 says I am v1:172.21.15.17:45916/0 (socket says 172.21.15.17:45916) 2024-03-16T10:12:29.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbffd23e640 1 -- 172.21.15.17:0/1889700166 learned_addr learned my addr 172.21.15.17:0/1889700166 (peer_addr_for_me v1:172.21.15.17:0/0) 2024-03-16T10:12:29.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbffda3f640 1 --2- 172.21.15.17:0/1889700166 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbff8145d70 0x7fbff8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:29.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 -- 172.21.15.17:0/1889700166 <== mon.0 v1:172.21.15.17:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4050430855 0 0) 0x7fbff806db30 con 0x7fbff806d750 2024-03-16T10:12:29.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 -- 172.21.15.17:0/1889700166 --> v1:172.21.15.17:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbfe4003530 con 0x7fbff806d750 2024-03-16T10:12:29.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 -- 172.21.15.17:0/1889700166 <== mon.0 v1:172.21.15.17:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2138018373 0 0) 0x7fbfe4003530 con 0x7fbff806d750 2024-03-16T10:12:29.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 -- 172.21.15.17:0/1889700166 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbff8145d70 msgr2=0x7fbff8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:29.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 --2- 172.21.15.17:0/1889700166 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbff8145d70 0x7fbff8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:29.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 -- 172.21.15.17:0/1889700166 >> v1:172.21.15.174:6789/0 conn(0x7fbff80affc0 legacy=0x7fbff8145660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:29.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 -- 172.21.15.17:0/1889700166 --> v1:172.21.15.17:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff80b4f10 con 0x7fbff806d750 2024-03-16T10:12:29.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 -- 172.21.15.17:0/1889700166 <== mon.0 v1:172.21.15.17:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3079593465 0 0) 0x7fbfec001680 con 0x7fbff806d750 2024-03-16T10:12:29.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 -- 172.21.15.17:0/1889700166 >> v1:172.21.15.17:6789/0 conn(0x7fbff806d750 legacy=0x7fbff80b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:29.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 --2- 172.21.15.17:0/1889700166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40039f0 0x7fbfe4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:29.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 --2- 172.21.15.17:0/1889700166 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbff8145d70 0x7fbfe4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:29.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 --2- 172.21.15.17:0/1889700166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfe4004d50 0x7fbfe40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:29.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 -- 172.21.15.17:0/1889700166 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbff8073380 con 0x7fbfe40039f0 2024-03-16T10:12:29.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 -- 172.21.15.17:0/1889700166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbfe400bb00 con 0x7fbfe4004d50 2024-03-16T10:12:29.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbfebfff640 1 -- 172.21.15.17:0/1889700166 --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fbfe400bd00 con 0x7fbff8145d70 2024-03-16T10:12:29.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbffd23e640 1 --2- 172.21.15.17:0/1889700166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40039f0 0x7fbfe4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:29.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbffca3d640 1 --2- 172.21.15.17:0/1889700166 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbff8145d70 0x7fbfe4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:29.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbffda3f640 1 --2- 172.21.15.17:0/1889700166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfe4004d50 0x7fbfe40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:29.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbffd23e640 1 -- 172.21.15.17:0/1889700166 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbff8145d70 msgr2=0x7fbfe4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:29.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbffd23e640 1 --2- 172.21.15.17:0/1889700166 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbff8145d70 0x7fbfe4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:29.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbffd23e640 1 -- 172.21.15.17:0/1889700166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfe4004d50 msgr2=0x7fbfe40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:29.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbffd23e640 1 --2- 172.21.15.17:0/1889700166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfe4004d50 0x7fbfe40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:29.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.823+0000 7fbffd23e640 1 -- 172.21.15.17:0/1889700166 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfe4003530 con 0x7fbfe40039f0 2024-03-16T10:12:29.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbffd23e640 1 --2- 172.21.15.17:0/1889700166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40039f0 0x7fbfe4003e10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbfec001060 tx=0x7fbfec01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46fde02c6f1ae1c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:29.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbfebfff640 1 -- 172.21.15.17:0/1889700166 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec007d40 con 0x7fbfe40039f0 2024-03-16T10:12:29.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbfebfff640 1 -- 172.21.15.17:0/1889700166 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfec0a9020 con 0x7fbfe40039f0 2024-03-16T10:12:29.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbfebfff640 1 -- 172.21.15.17:0/1889700166 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfec005b00 con 0x7fbfe40039f0 2024-03-16T10:12:29.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1889700166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40039f0 msgr2=0x7fbfe4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:29.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbfff4a2640 1 --2- 172.21.15.17:0/1889700166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfe40039f0 0x7fbfe4003e10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbfec001060 tx=0x7fbfec01cba0 comp rx=0 tx=0).stop 2024-03-16T10:12:29.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbffda3f640 1 -- 172.21.15.17:0/1889700166 reap_dead start 2024-03-16T10:12:29.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1889700166 shutdown_connections 2024-03-16T10:12:29.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1889700166 >> 172.21.15.17:0/1889700166 conn(0x7fbff80b55c0 msgr2=0x7fbff80b59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:29.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1889700166 shutdown_connections 2024-03-16T10:12:29.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1889700166 wait complete. 2024-03-16T10:12:29.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbfff4a2640 1 Processor -- start 2024-03-16T10:12:29.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbfff4a2640 1 -- start start 2024-03-16T10:12:29.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbfff4a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfe40039f0 0x7fbff8070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:29.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbfff4a2640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbff8146bc0 0x7fbff8139040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:29.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.827+0000 7fbfff4a2640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbff8147040 0x7fbff8139580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:29.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbfff4a2640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbff8157d00 con 0x7fbff8147040 2024-03-16T10:12:29.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbfff4a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbff8157b80 con 0x7fbfe40039f0 2024-03-16T10:12:29.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbfff4a2640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7fbff8157e80 con 0x7fbff8146bc0 2024-03-16T10:12:29.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbffda3f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbff8147040 0x7fbff8139580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:29.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbffda3f640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbff8147040 0x7fbff8139580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37306/0 (socket says 172.21.15.17:37306) 2024-03-16T10:12:29.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbffda3f640 1 -- 172.21.15.17:0/1107289404 learned_addr learned my addr 172.21.15.17:0/1107289404 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:29.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbffca3d640 1 --2- 172.21.15.17:0/1107289404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbff8146bc0 0x7fbff8139040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:29.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbffd23e640 1 --2- 172.21.15.17:0/1107289404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfe40039f0 0x7fbff8070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:29.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbffda3f640 1 -- 172.21.15.17:0/1107289404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbff8146bc0 msgr2=0x7fbff8139040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:29.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbffda3f640 1 --2- 172.21.15.17:0/1107289404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbff8146bc0 0x7fbff8139040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:29.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbffda3f640 1 -- 172.21.15.17:0/1107289404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfe40039f0 msgr2=0x7fbff8070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbffda3f640 1 --2- 172.21.15.17:0/1107289404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfe40039f0 0x7fbff8070eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbffda3f640 1 -- 172.21.15.17:0/1107289404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbff806fe70 con 0x7fbff8147040 2024-03-16T10:12:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbffda3f640 1 --2- 172.21.15.17:0/1107289404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbff8147040 0x7fbff8139580 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbff80b26e0 tx=0x7fbff0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbfe9ffb640 1 -- 172.21.15.17:0/1107289404 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0019020 con 0x7fbff8147040 2024-03-16T10:12:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbfe9ffb640 1 -- 172.21.15.17:0/1107289404 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbff0003620 con 0x7fbff8147040 2024-03-16T10:12:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1107289404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbff813fb30 con 0x7fbff8147040 2024-03-16T10:12:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbfe9ffb640 1 -- 172.21.15.17:0/1107289404 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff0003a50 con 0x7fbff8147040 2024-03-16T10:12:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1107289404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbff80af720 con 0x7fbff8147040 2024-03-16T10:12:29.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1107289404 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff8068360 con 0x7fbff8147040 2024-03-16T10:12:29.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbfe9ffb640 1 -- 172.21.15.17:0/1107289404 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fbff001c020 con 0x7fbff8147040 2024-03-16T10:12:29.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbfe9ffb640 1 --2- 172.21.15.17:0/1107289404 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fbfdc067660 0x7fbfdc069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:29.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbfe9ffb640 1 -- 172.21.15.17:0/1107289404 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fbff0085690 con 0x7fbff8147040 2024-03-16T10:12:29.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.831+0000 7fbffd23e640 1 --2- 172.21.15.17:0/1107289404 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fbfdc067660 0x7fbfdc069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:29.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.835+0000 7fbffd23e640 1 --2- 172.21.15.17:0/1107289404 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fbfdc067660 0x7fbfdc069b20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0aa000 tx=0x7fbfec000ff0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:29.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.835+0000 7fbfe9ffb640 1 -- 172.21.15.17:0/1107289404 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbff0023030 con 0x7fbff8147040 2024-03-16T10:12:29.975 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T10:12:29.978 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T10:12:29.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.995+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1107289404 --> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbff806a130 con 0x7fbfdc067660 2024-03-16T10:12:29.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.995+0000 7fbfe9ffb640 1 -- 172.21.15.17:0/1107289404 <== mgr.4099 v2:172.21.15.174:6832/2142122996 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13355 (secure 0 0 0) 0x7fbff806a130 con 0x7fbfdc067660 2024-03-16T10:12:29.999 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:12:29.999 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-03-16T10:12:30.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.999+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1107289404 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fbfdc067660 msgr2=0x7fbfdc069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.999+0000 7fbfff4a2640 1 --2- 172.21.15.17:0/1107289404 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fbfdc067660 0x7fbfdc069b20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0aa000 tx=0x7fbfec000ff0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.999+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1107289404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbff8147040 msgr2=0x7fbff8139580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.999+0000 7fbfff4a2640 1 --2- 172.21.15.17:0/1107289404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbff8147040 0x7fbff8139580 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbff80b26e0 tx=0x7fbff0011920 comp rx=0 tx=0).stop 2024-03-16T10:12:30.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.999+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1107289404 shutdown_connections 2024-03-16T10:12:30.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.999+0000 7fbfff4a2640 1 --2- 172.21.15.17:0/1107289404 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7fbfdc067660 0x7fbfdc069b20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.999+0000 7fbfff4a2640 1 --2- 172.21.15.17:0/1107289404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbff8147040 0x7fbff8139580 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.999+0000 7fbfff4a2640 1 --2- 172.21.15.17:0/1107289404 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7fbff8146bc0 0x7fbff8139040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.999+0000 7fbfff4a2640 1 --2- 172.21.15.17:0/1107289404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfe40039f0 0x7fbff8070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.999+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1107289404 >> 172.21.15.17:0/1107289404 conn(0x7fbff80b55c0 msgr2=0x7fbff806c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:30.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.999+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1107289404 shutdown_connections 2024-03-16T10:12:30.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:29.999+0000 7fbfff4a2640 1 -- 172.21.15.17:0/1107289404 wait complete. 2024-03-16T10:12:30.018 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-03-16T10:12:28.043766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236928,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737792,"statfs":{"total":773094113280,"available":772851499008,"internally_reserved":0,"allocated":4235264,"data_stored":2351392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003450"},"pg_stats":[{"pgid":"1.0","version":"33'119","reported_seq":204,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-16T10:12:09.440384+0000","last_change":"2024-03-16T10:12:09.440471+0000","last_active":"2024-03-16T10:12:09.440384+0000","last_peered":"2024-03-16T10:12:09.440384+0000","last_clean":"2024-03-16T10:12:09.440384+0000","last_became_active":"2024-03-16T10:10:48.243054+0000","last_became_peered":"2024-03-16T10:10:48.243054+0000","last_unstale":"2024-03-16T10:12:09.440384+0000","last_undegraded":"2024-03-16T10:12:09.440384+0000","last_fullsized":"2024-03-16T10:12:09.440384+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'119","last_scrub_stamp":"2024-03-16T10:12:09.440187+0000","last_deep_scrub":"33'119","last_deep_scrub_stamp":"2024-03-16T10:12:09.440187+0000","last_clean_scrub_stamp":"2024-03-16T10:12:09.440187+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T17:52:47.274660+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":974848,"data_stored":736700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":974848,"data_stored":736700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T10:12:30.019 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-16T10:12:30.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.183+0000 7f2399b01640 1 Processor -- start 2024-03-16T10:12:30.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.183+0000 7f2399b01640 1 -- start start 2024-03-16T10:12:30.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.183+0000 7f2399b01640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f23940ae060 0x7f2394072b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:30.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.183+0000 7f2399b01640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23940730c0 0x7f239414fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:30.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.183+0000 7f2399b01640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23940af0c0 con 0x7f23940b2400 2024-03-16T10:12:30.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.183+0000 7f2399b01640 1 -- --> v2:172.21.15.174:3300/0 -- mon_getmap magic: 0 -- 0x7f23940afcf0 con 0x7f23940ae060 2024-03-16T10:12:30.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.183+0000 7f2399b01640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f23940b3fe0 con 0x7f23940730c0 2024-03-16T10:12:30.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.187+0000 7f2393fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23940730c0 0x7f239414fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:30.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.187+0000 7f238bfff640 1 --2- >> v2:172.21.15.174:3300/0 conn(0x7f23940ae060 0x7f2394072b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:30.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.187+0000 7f2393fff640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23940730c0 0x7f239414fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3301/0 says I am v2:172.21.15.17:43608/0 (socket says 172.21.15.17:43608) 2024-03-16T10:12:30.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.187+0000 7f2393fff640 1 -- 172.21.15.17:0/1080577804 learned_addr learned my addr 172.21.15.17:0/1080577804 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:30.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.187+0000 7f238bfff640 1 -- 172.21.15.17:0/1080577804 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23940730c0 msgr2=0x7f239414fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T10:12:30.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.187+0000 7f238bfff640 1 --2- 172.21.15.17:0/1080577804 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23940730c0 0x7f239414fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.187+0000 7f238bfff640 1 -- 172.21.15.17:0/1080577804 >> v1:172.21.15.174:6789/0 conn(0x7f23940b2400 legacy=0x7f23941452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.187+0000 7f238bfff640 1 -- 172.21.15.17:0/1080577804 --> v2:172.21.15.174:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23940b27e0 con 0x7f23940ae060 2024-03-16T10:12:30.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.187+0000 7f2393fff640 1 --2- 172.21.15.17:0/1080577804 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23940730c0 0x7f239414fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T10:12:30.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.187+0000 7f238bfff640 1 --2- 172.21.15.17:0/1080577804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23940ae060 0x7f2394072b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f237c0097c0 tx=0x7f237c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b3981242bb71ad server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:30.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.187+0000 7f2392ffd640 1 -- 172.21.15.17:0/1080577804 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c0aa620 con 0x7f23940ae060 2024-03-16T10:12:30.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.187+0000 7f2392ffd640 1 -- 172.21.15.17:0/1080577804 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f237c0af070 con 0x7f23940ae060 2024-03-16T10:12:30.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.187+0000 7f2392ffd640 1 -- 172.21.15.17:0/1080577804 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c0aa9e0 con 0x7f23940ae060 2024-03-16T10:12:30.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 -- 172.21.15.17:0/1080577804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23940ae060 msgr2=0x7f2394072b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 --2- 172.21.15.17:0/1080577804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23940ae060 0x7f2394072b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f237c0097c0 tx=0x7f237c01c4a0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 -- 172.21.15.17:0/1080577804 shutdown_connections 2024-03-16T10:12:30.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 --2- 172.21.15.17:0/1080577804 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23940730c0 0x7f239414fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 --2- 172.21.15.17:0/1080577804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23940ae060 0x7f2394072b80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 -- 172.21.15.17:0/1080577804 >> 172.21.15.17:0/1080577804 conn(0x7f23940b8e90 msgr2=0x7f23940b92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:30.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 -- 172.21.15.17:0/1080577804 shutdown_connections 2024-03-16T10:12:30.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 -- 172.21.15.17:0/1080577804 wait complete. 2024-03-16T10:12:30.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 Processor -- start 2024-03-16T10:12:30.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 -- start start 2024-03-16T10:12:30.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23940730c0 0x7f2394139590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:30.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23940ae060 0x7f23941e1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:30.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23940b2400 0x7f2394132820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:30.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2394152aa0 con 0x7f23940ae060 2024-03-16T10:12:30.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2394152920 con 0x7f23940b2400 2024-03-16T10:12:30.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f2394152c20 con 0x7f23940730c0 2024-03-16T10:12:30.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f238bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23940ae060 0x7f23941e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:30.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f238bfff640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23940ae060 0x7f23941e1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37314/0 (socket says 172.21.15.17:37314) 2024-03-16T10:12:30.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f238bfff640 1 -- 172.21.15.17:0/2560161812 learned_addr learned my addr 172.21.15.17:0/2560161812 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:30.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2393fff640 1 --2- 172.21.15.17:0/2560161812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23940b2400 0x7f2394132820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:30.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f23937fe640 1 --2- 172.21.15.17:0/2560161812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23940730c0 0x7f2394139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:30.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f238bfff640 1 -- 172.21.15.17:0/2560161812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23940730c0 msgr2=0x7f2394139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f238bfff640 1 --2- 172.21.15.17:0/2560161812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23940730c0 0x7f2394139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f238bfff640 1 -- 172.21.15.17:0/2560161812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23940b2400 msgr2=0x7f2394132820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f238bfff640 1 --2- 172.21.15.17:0/2560161812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23940b2400 0x7f2394132820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f238bfff640 1 -- 172.21.15.17:0/2560161812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23941362b0 con 0x7f23940ae060 2024-03-16T10:12:30.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f238bfff640 1 --2- 172.21.15.17:0/2560161812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23940ae060 0x7f23941e1180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f237c0ac040 tx=0x7f237c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:30.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2390ff9640 1 -- 172.21.15.17:0/2560161812 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c0aac70 con 0x7f23940ae060 2024-03-16T10:12:30.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2390ff9640 1 -- 172.21.15.17:0/2560161812 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f237c0aae10 con 0x7f23940ae060 2024-03-16T10:12:30.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 -- 172.21.15.17:0/2560161812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2394137060 con 0x7f23940ae060 2024-03-16T10:12:30.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2390ff9640 1 -- 172.21.15.17:0/2560161812 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c0be650 con 0x7f23940ae060 2024-03-16T10:12:30.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.191+0000 7f2399b01640 1 -- 172.21.15.17:0/2560161812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2394138060 con 0x7f23940ae060 2024-03-16T10:12:30.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.195+0000 7f2399b01640 1 -- 172.21.15.17:0/2560161812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2394068360 con 0x7f23940ae060 2024-03-16T10:12:30.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.195+0000 7f2390ff9640 1 -- 172.21.15.17:0/2560161812 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f237c0bc020 con 0x7f23940ae060 2024-03-16T10:12:30.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.195+0000 7f2390ff9640 1 --2- 172.21.15.17:0/2560161812 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2374067660 0x7f2374069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:30.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.195+0000 7f2390ff9640 1 -- 172.21.15.17:0/2560161812 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f237c11f5f0 con 0x7f23940ae060 2024-03-16T10:12:30.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.195+0000 7f23937fe640 1 --2- 172.21.15.17:0/2560161812 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2374067660 0x7f2374069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:30.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.195+0000 7f23937fe640 1 --2- 172.21.15.17:0/2560161812 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2374067660 0x7f2374069b20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2394139e80 tx=0x7f2384009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:30.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.199+0000 7f2390ff9640 1 -- 172.21.15.17:0/2560161812 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f237c016090 con 0x7f23940ae060 2024-03-16T10:12:30.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.359+0000 7f2399b01640 1 -- 172.21.15.17:0/2560161812 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f23940008e0 con 0x7f23940ae060 2024-03-16T10:12:30.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.367+0000 7f2390ff9640 1 -- 172.21.15.17:0/2560161812 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f237c0f00d0 con 0x7f23940ae060 2024-03-16T10:12:30.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.367+0000 7f2390ff9640 1 -- 172.21.15.17:0/2560161812 <== mon.0 v2:172.21.15.17:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f237c0f4170 con 0x7f23940ae060 2024-03-16T10:12:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.367+0000 7f2399b01640 1 -- 172.21.15.17:0/2560161812 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2374067660 msgr2=0x7f2374069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.367+0000 7f2399b01640 1 --2- 172.21.15.17:0/2560161812 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2374067660 0x7f2374069b20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2394139e80 tx=0x7f2384009040 comp rx=0 tx=0).stop 2024-03-16T10:12:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.367+0000 7f2399b01640 1 -- 172.21.15.17:0/2560161812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23940ae060 msgr2=0x7f23941e1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.367+0000 7f2399b01640 1 --2- 172.21.15.17:0/2560161812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23940ae060 0x7f23941e1180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f237c0ac040 tx=0x7f237c01ace0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.367+0000 7f2399b01640 1 -- 172.21.15.17:0/2560161812 shutdown_connections 2024-03-16T10:12:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.367+0000 7f2399b01640 1 --2- 172.21.15.17:0/2560161812 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f2374067660 0x7f2374069b20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.367+0000 7f2399b01640 1 --2- 172.21.15.17:0/2560161812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23940b2400 0x7f2394132820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.367+0000 7f2399b01640 1 --2- 172.21.15.17:0/2560161812 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23940ae060 0x7f23941e1180 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.367+0000 7f2399b01640 1 --2- 172.21.15.17:0/2560161812 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f23940730c0 0x7f2394139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.367+0000 7f2399b01640 1 -- 172.21.15.17:0/2560161812 >> 172.21.15.17:0/2560161812 conn(0x7f23940b8e90 msgr2=0x7f2394145a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.371+0000 7f2399b01640 1 -- 172.21.15.17:0/2560161812 shutdown_connections 2024-03-16T10:12:30.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.371+0000 7f2399b01640 1 -- 172.21.15.17:0/2560161812 wait complete. 2024-03-16T10:12:30.388 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-16T10:12:30.388 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T10:12:30.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.507+0000 7f06bf17d640 1 Processor -- start 2024-03-16T10:12:30.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.507+0000 7f06bf17d640 1 -- start start 2024-03-16T10:12:30.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.507+0000 7f06bf17d640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06b80731d0 0x7f06b80b9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:30.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.507+0000 7f06bf17d640 1 -- --> v1:172.21.15.17:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b8145eb0 con 0x7f06b80b0e40 2024-03-16T10:12:30.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.507+0000 7f06bf17d640 1 -- --> v1:172.21.15.174:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b80b7a90 con 0x7f06b80b76b0 2024-03-16T10:12:30.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.507+0000 7f06bf17d640 1 -- --> v2:172.21.15.17:3300/0 -- mon_getmap magic: 0 -- 0x7f06b80b9100 con 0x7f06b80731d0 2024-03-16T10:12:30.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.507+0000 7f06b7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06b80731d0 0x7f06b80b9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:30.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.507+0000 7f06b7fff640 1 --2- >> v2:172.21.15.17:3300/0 conn(0x7f06b80731d0 0x7f06b80b9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37320/0 (socket says 172.21.15.17:37320) 2024-03-16T10:12:30.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.507+0000 7f06b7fff640 1 -- 172.21.15.17:0/2430487180 learned_addr learned my addr 172.21.15.17:0/2430487180 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.507+0000 7f06b7fff640 1 -- 172.21.15.17:0/2430487180 >> v1:172.21.15.174:6789/0 conn(0x7f06b80b76b0 legacy=0x7f06b81457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.507+0000 7f06b7fff640 1 -- 172.21.15.17:0/2430487180 >> v1:172.21.15.17:6789/0 conn(0x7f06b80b0e40 legacy=0x7f06b8156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.507+0000 7f06b7fff640 1 -- 172.21.15.17:0/2430487180 --> v2:172.21.15.17:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b80b96c0 con 0x7f06b80731d0 2024-03-16T10:12:30.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06b7fff640 1 --2- 172.21.15.17:0/2430487180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06b80731d0 0x7f06b80b9ec0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f06a8009870 tx=0x7f06a801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8241f45be71dc61a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:30.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06b77fe640 1 -- 172.21.15.17:0/2430487180 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a80b1070 con 0x7f06b80731d0 2024-03-16T10:12:30.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06b77fe640 1 -- 172.21.15.17:0/2430487180 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f06a80ac600 con 0x7f06b80731d0 2024-03-16T10:12:30.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06b77fe640 1 -- 172.21.15.17:0/2430487180 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a80ac9f0 con 0x7f06b80731d0 2024-03-16T10:12:30.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 -- 172.21.15.17:0/2430487180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06b80731d0 msgr2=0x7f06b80b9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 --2- 172.21.15.17:0/2430487180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06b80731d0 0x7f06b80b9ec0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f06a8009870 tx=0x7f06a801c4d0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 -- 172.21.15.17:0/2430487180 shutdown_connections 2024-03-16T10:12:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 --2- 172.21.15.17:0/2430487180 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06b80731d0 0x7f06b80b9ec0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 -- 172.21.15.17:0/2430487180 >> 172.21.15.17:0/2430487180 conn(0x7f06b80baac0 msgr2=0x7f06b80baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 -- 172.21.15.17:0/2430487180 shutdown_connections 2024-03-16T10:12:30.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 -- 172.21.15.17:0/2430487180 wait complete. 2024-03-16T10:12:30.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 Processor -- start 2024-03-16T10:12:30.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 -- start start 2024-03-16T10:12:30.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06b80b0e40 0x7f06b813d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:30.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 --2- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06b80b76b0 0x7f06b8146ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:30.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06b8147000 0x7f06b80b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:30.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f06b815a640 con 0x7f06b8147000 2024-03-16T10:12:30.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06b815a4c0 con 0x7f06b80b0e40 2024-03-16T10:12:30.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bf17d640 1 -- --> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] -- mon_getmap magic: 0 -- 0x7f06b815a7c0 con 0x7f06b80b76b0 2024-03-16T10:12:30.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bd71a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06b8147000 0x7f06b80b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:30.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bd71a640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06b8147000 0x7f06b80b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37330/0 (socket says 172.21.15.17:37330) 2024-03-16T10:12:30.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bd71a640 1 -- 172.21.15.17:0/3152466821 learned_addr learned my addr 172.21.15.17:0/3152466821 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-03-16T10:12:30.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06b7fff640 1 --2- 172.21.15.17:0/3152466821 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06b80b76b0 0x7f06b8146ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:30.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bcf19640 1 --2- 172.21.15.17:0/3152466821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06b80b0e40 0x7f06b813d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:30.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bd71a640 1 -- 172.21.15.17:0/3152466821 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06b80b76b0 msgr2=0x7f06b8146ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bd71a640 1 --2- 172.21.15.17:0/3152466821 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06b80b76b0 0x7f06b8146ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bd71a640 1 -- 172.21.15.17:0/3152466821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06b80b0e40 msgr2=0x7f06b813d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bd71a640 1 --2- 172.21.15.17:0/3152466821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06b80b0e40 0x7f06b813d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.511+0000 7f06bd71a640 1 -- 172.21.15.17:0/3152466821 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b81474b0 con 0x7f06b8147000 2024-03-16T10:12:30.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.515+0000 7f06bd71a640 1 --2- 172.21.15.17:0/3152466821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06b8147000 0x7f06b80b2e20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f06b000c430 tx=0x7f06b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:30.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.515+0000 7f06b57fa640 1 -- 172.21.15.17:0/3152466821 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0005220 con 0x7f06b8147000 2024-03-16T10:12:30.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.515+0000 7f06b57fa640 1 -- 172.21.15.17:0/3152466821 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f06b00053c0 con 0x7f06b8147000 2024-03-16T10:12:30.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.515+0000 7f06bf17d640 1 -- 172.21.15.17:0/3152466821 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b8146630 con 0x7f06b8147000 2024-03-16T10:12:30.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.515+0000 7f06b57fa640 1 -- 172.21.15.17:0/3152466821 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b0007290 con 0x7f06b8147000 2024-03-16T10:12:30.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.515+0000 7f06bf17d640 1 -- 172.21.15.17:0/3152466821 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b813dab0 con 0x7f06b8147000 2024-03-16T10:12:30.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.515+0000 7f06bf17d640 1 -- 172.21.15.17:0/3152466821 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b813dce0 con 0x7f06b8147000 2024-03-16T10:12:30.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.515+0000 7f06b57fa640 1 -- 172.21.15.17:0/3152466821 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f06b0019020 con 0x7f06b8147000 2024-03-16T10:12:30.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.515+0000 7f06b57fa640 1 --2- 172.21.15.17:0/3152466821 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f069c067660 0x7f069c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T10:12:30.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.515+0000 7f06b57fa640 1 -- 172.21.15.17:0/3152466821 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f06b0084930 con 0x7f06b8147000 2024-03-16T10:12:30.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.515+0000 7f06bcf19640 1 --2- 172.21.15.17:0/3152466821 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f069c067660 0x7f069c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T10:12:30.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.515+0000 7f06bcf19640 1 --2- 172.21.15.17:0/3152466821 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f069c067660 0x7f069c069b20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f06b80a0d70 tx=0x7f06a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T10:12:30.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.519+0000 7f06b57fa640 1 -- 172.21.15.17:0/3152466821 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f06b00558a0 con 0x7f06b8147000 2024-03-16T10:12:30.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.715+0000 7f06bf17d640 1 -- 172.21.15.17:0/3152466821 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06b806a130 con 0x7f06b8147000 2024-03-16T10:12:30.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.715+0000 7f06b57fa640 1 -- 172.21.15.17:0/3152466821 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+2222 (secure 0 0 0) 0x7f06b00551a0 con 0x7f06b8147000 2024-03-16T10:12:30.719 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-03-16T10:12:30.720 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4738,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1674220578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1674220578},{"type":"v1","addr":"172.21.15.174:6835","nonce":1674220578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3480063095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3480063095},{"type":"v1","addr":"172.21.15.17:6833","nonce":3480063095}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[]} 2024-03-16T10:12:30.720 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 17 2024-03-16T10:12:30.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.719+0000 7f06bf17d640 1 -- 172.21.15.17:0/3152466821 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f069c067660 msgr2=0x7f069c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.719+0000 7f06bf17d640 1 --2- 172.21.15.17:0/3152466821 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f069c067660 0x7f069c069b20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f06b80a0d70 tx=0x7f06a0009040 comp rx=0 tx=0).stop 2024-03-16T10:12:30.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.719+0000 7f06bf17d640 1 -- 172.21.15.17:0/3152466821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06b8147000 msgr2=0x7f06b80b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T10:12:30.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.719+0000 7f06bf17d640 1 --2- 172.21.15.17:0/3152466821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06b8147000 0x7f06b80b2e20 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f06b000c430 tx=0x7f06b000c8f0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.719+0000 7f06bf17d640 1 -- 172.21.15.17:0/3152466821 shutdown_connections 2024-03-16T10:12:30.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.719+0000 7f06bf17d640 1 --2- 172.21.15.17:0/3152466821 >> [v2:172.21.15.174:6832/2142122996,v1:172.21.15.174:6833/2142122996] conn(0x7f069c067660 0x7f069c069b20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.719+0000 7f06bf17d640 1 --2- 172.21.15.17:0/3152466821 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06b8147000 0x7f06b80b2e20 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.719+0000 7f06bf17d640 1 --2- 172.21.15.17:0/3152466821 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f06b80b76b0 0x7f06b8146ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.719+0000 7f06bf17d640 1 --2- 172.21.15.17:0/3152466821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06b80b0e40 0x7f06b813d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T10:12:30.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.719+0000 7f06bf17d640 1 -- 172.21.15.17:0/3152466821 >> 172.21.15.17:0/3152466821 conn(0x7f06b80baac0 msgr2=0x7f06b80afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T10:12:30.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.719+0000 7f06bf17d640 1 -- 172.21.15.17:0/3152466821 shutdown_connections 2024-03-16T10:12:30.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-03-16T10:12:30.719+0000 7f06bf17d640 1 -- 172.21.15.17:0/3152466821 wait complete. 2024-03-16T10:12:30.738 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-16T10:12:30.738 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-16T10:12:30.739 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-16T10:12:30.739 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:30.748 INFO:tasks.ceph.mds.a:Stopped 2024-03-16T10:12:30.749 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-16T10:12:30.749 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:30.766 INFO:tasks.ceph.mds.b:Stopped 2024-03-16T10:12:30.767 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-16T10:12:30.767 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-16T10:12:30.767 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:35.190 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-03-16T10:12:36.869 INFO:tasks.ceph.osd.0:Stopped 2024-03-16T10:12:36.869 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-16T10:12:36.869 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:36.933 INFO:tasks.ceph.osd.1:Stopped 2024-03-16T10:12:36.934 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-16T10:12:36.934 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:37.028 INFO:tasks.ceph.osd.2:Stopped 2024-03-16T10:12:37.028 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-16T10:12:37.028 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:37.124 INFO:tasks.ceph.osd.3:Stopped 2024-03-16T10:12:37.125 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-16T10:12:37.125 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:37.203 INFO:tasks.ceph.osd.4:Stopped 2024-03-16T10:12:37.203 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-16T10:12:37.203 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:37.282 INFO:tasks.ceph.osd.5:Stopped 2024-03-16T10:12:37.283 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-16T10:12:37.283 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:37.374 INFO:tasks.ceph.osd.6:Stopped 2024-03-16T10:12:37.375 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-16T10:12:37.375 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:37.461 INFO:tasks.ceph.osd.7:Stopped 2024-03-16T10:12:37.461 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-16T10:12:37.462 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-16T10:12:37.462 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:37.489 INFO:tasks.ceph.mgr.y:Stopped 2024-03-16T10:12:37.490 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-16T10:12:37.490 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:37.519 INFO:tasks.ceph.mgr.x:Stopped 2024-03-16T10:12:37.519 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-16T10:12:37.519 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-16T10:12:37.519 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:37.551 INFO:tasks.ceph.mon.a:Stopped 2024-03-16T10:12:37.551 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-16T10:12:37.551 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:37.580 INFO:tasks.ceph.mon.b:Stopped 2024-03-16T10:12:37.580 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-16T10:12:37.580 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T10:12:37.643 INFO:tasks.ceph.mon.c:Stopped 2024-03-16T10:12:37.643 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-16T10:12:37.643 DEBUG:teuthology.orchestra.run.smithi017:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-16T10:12:37.661 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-16T10:12:37.661 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-16T10:12:37.923 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-16T10:12:37.924 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-16T10:12:38.090 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-16T10:12:38.090 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-16T10:12:38.263 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi017.front.sepia.ceph.com 2024-03-16T10:12:38.263 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-16T10:12:38.456 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi174.front.sepia.ceph.com 2024-03-16T10:12:38.456 DEBUG:teuthology.orchestra.run.smithi174:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-16T10:12:38.926 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi174.front.sepia.ceph.com 2024-03-16T10:12:38.926 DEBUG:teuthology.orchestra.run.smithi174:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-16T10:12:39.117 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi174.front.sepia.ceph.com 2024-03-16T10:12:39.117 DEBUG:teuthology.orchestra.run.smithi174:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-16T10:12:39.299 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi174.front.sepia.ceph.com 2024-03-16T10:12:39.299 DEBUG:teuthology.orchestra.run.smithi174:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-16T10:12:39.493 INFO:tasks.ceph:Archiving mon data... 2024-03-16T10:12:39.494 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360/data/mon.a.tgz 2024-03-16T10:12:39.494 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2024-03-16T10:12:39.500 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.ZoTuP2Nw2o 2024-03-16T10:12:39.501 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ZoTuP2Nw2o 2024-03-16T10:12:39.616 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.ZoTuP2Nw2o 2024-03-16T10:12:39.684 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.ZoTuP2Nw2o is 270KB 2024-03-16T10:12:39.745 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.ZoTuP2Nw2o 2024-03-16T10:12:39.751 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360/data/mon.b.tgz 2024-03-16T10:12:39.751 DEBUG:teuthology.orchestra.run.smithi174:> mktemp 2024-03-16T10:12:39.757 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.Gr6k1wkqaX 2024-03-16T10:12:39.757 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Gr6k1wkqaX 2024-03-16T10:12:39.882 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.Gr6k1wkqaX 2024-03-16T10:12:39.944 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.Gr6k1wkqaX is 363KB 2024-03-16T10:12:40.001 DEBUG:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.Gr6k1wkqaX 2024-03-16T10:12:40.006 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360/data/mon.c.tgz 2024-03-16T10:12:40.007 DEBUG:teuthology.orchestra.run.smithi174:> mktemp 2024-03-16T10:12:40.051 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.cwyMzP26sF 2024-03-16T10:12:40.052 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.cwyMzP26sF 2024-03-16T10:12:40.177 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.cwyMzP26sF 2024-03-16T10:12:40.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-03-16T10:12:40.240 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.cwyMzP26sF is 363KB 2024-03-16T10:12:40.301 DEBUG:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.cwyMzP26sF 2024-03-16T10:12:40.306 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-16T10:12:40.307 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-16T10:12:40.309 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-16T10:12:40.369 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-16T10:12:40.411 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-16T10:12:40.412 INFO:tasks.ceph:Archiving crash dumps... 2024-03-16T10:12:40.413 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360/remote/smithi017/crash 2024-03-16T10:12:40.414 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-16T10:12:40.428 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360/remote/smithi174/crash 2024-03-16T10:12:40.429 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-16T10:12:40.467 INFO:tasks.ceph:Shutting down logrotate 2024-03-16T10:12:40.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T10:12:40.473 DEBUG:teuthology.orchestra.run.smithi174:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T10:12:40.556 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-16T10:12:40.568 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-16T10:12:40.580 INFO:tasks.ceph:Compressing logs... 2024-03-16T10:12:40.580 DEBUG:teuthology.orchestra.run.smithi017:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T10:12:40.617 DEBUG:teuthology.orchestra.run.smithi174:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T10:12:40.636 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32240.log 2024-03-16T10:12:40.636 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32848.log 2024-03-16T10:12:40.637 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36705.log 2024-03-16T10:12:40.638 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4%/var/log/ceph/ceph-client.admin.32848.log: -- replaced with /var/log/ceph/ceph-client.admin.32240.log.gz 2024-03-16T10:12:40.638 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35939.log 2024-03-16T10:12:40.638 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32848.log.gz 2024-03-16T10:12:40.638 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34770.log 2024-03-16T10:12:40.640 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33091.log 2024-03-16T10:12:40.640 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36705.log.gz 2024-03-16T10:12:40.641 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34770.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35939.log.gz 2024-03-16T10:12:40.641 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35204.log 2024-03-16T10:12:40.641 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34770.log.gz 2024-03-16T10:12:40.641 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32838.log 2024-03-16T10:12:40.641 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34133.log 2024-03-16T10:12:40.641 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33091.log.gz 2024-03-16T10:12:40.641 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32838.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-16T10:12:40.641 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35204.log.gz 2024-03-16T10:12:40.641 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32838.log.gz 2024-03-16T10:12:40.642 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35100.log 2024-03-16T10:12:40.645 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mgr.y.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34133.log.gz 2024-03-16T10:12:40.645 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35637.log 2024-03-16T10:12:40.645 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32467.log 2024-03-16T10:12:40.645 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36679.log 2024-03-16T10:12:40.645 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35100.log.gz 2024-03-16T10:12:40.646 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35637.log.gz 2024-03-16T10:12:40.646 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27246.log 2024-03-16T10:12:40.646 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32467.log.gz 2024-03-16T10:12:40.646 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34348.log 2024-03-16T10:12:40.646 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36679.log: gzip/var/log/ceph/ceph-client.admin.27246.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35429.log 2024-03-16T10:12:40.646 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36679.log.gz 2024-03-16T10:12:40.646 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36549.log 2024-03-16T10:12:40.646 INFO:teuthology.orchestra.run.smithi017.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.27246.log.gz 2024-03-16T10:12:40.647 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35429.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34348.log.gz 2024-03-16T10:12:40.647 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-16T10:12:40.647 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35429.log.gz 2024-03-16T10:12:40.647 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35913.log 2024-03-16T10:12:40.647 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35022.log 2024-03-16T10:12:40.647 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36549.log.gz 2024-03-16T10:12:40.647 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34270.log 2024-03-16T10:12:40.647 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35913.log.gz 2024-03-16T10:12:40.660 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-16T10:12:40.660 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-16T10:12:40.660 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27525.log 2024-03-16T10:12:40.660 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27549.log 2024-03-16T10:12:40.661 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.admin.27525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27597.log 2024-03-16T10:12:40.661 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.admin.27549.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27525.log.gz 2024-03-16T10:12:40.661 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-16T10:12:40.661 INFO:teuthology.orchestra.run.smithi174.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27549.log.gz 2024-03-16T10:12:40.661 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.admin.27597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27453.log 2024-03-16T10:12:40.661 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 88.6% 2024-03-16T10:12:40.661 INFO:teuthology.orchestra.run.smithi174.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27597.log.gz 2024-03-16T10:12:40.661 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.admin.27453.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-16T10:12:40.661 INFO:teuthology.orchestra.run.smithi174.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27453.log.gz 2024-03-16T10:12:40.662 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27573.log 2024-03-16T10:12:40.662 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-16T10:12:40.662 INFO:teuthology.orchestra.run.smithi174.stderr:gzip/var/log/ceph/ceph-client.admin.27573.log: -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-16T10:12:40.662 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-mon.c.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27573.log.gz 2024-03-16T10:12:40.662 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27621.log 2024-03-16T10:12:40.662 INFO:teuthology.orchestra.run.smithi174.stderr: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-16T10:12:40.662 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-16T10:12:40.662 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.27621.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-16T10:12:40.662 INFO:teuthology.orchestra.run.smithi174.stderr: 89.5%/var/log/ceph/ceph-mds.b.log: -- replaced with /var/log/ceph/ceph-client.admin.27621.log.gz 2024-03-16T10:12:40.663 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-16T10:12:40.663 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-mon.b.log: 91.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-16T10:12:40.663 INFO:teuthology.orchestra.run.smithi174.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-16T10:12:40.663 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27477.log 2024-03-16T10:12:40.663 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27501.log 2024-03-16T10:12:40.663 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.admin.27477.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27477.log.gz 2024-03-16T10:12:40.663 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/ceph-client.admin.27501.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27501.log.gz 2024-03-16T10:12:40.678 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36497.log 2024-03-16T10:12:40.678 INFO:teuthology.orchestra.run.smithi017.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-16T10:12:40.678 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35022.log.gz/var/log/ceph/ceph-client.admin.34270.log: 2024-03-16T10:12:40.678 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34796.log 2024-03-16T10:12:40.678 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34270.log.gz 2024-03-16T10:12:40.678 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35152.log 2024-03-16T10:12:40.678 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36497.log.gz 2024-03-16T10:12:40.678 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33143.log 2024-03-16T10:12:40.679 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34796.log: /var/log/ceph/ceph-client.admin.35152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36100.log 2024-03-16T10:12:40.679 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34796.log.gz 2024-03-16T10:12:40.679 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35152.log.gz 2024-03-16T10:12:40.679 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34296.log 2024-03-16T10:12:40.679 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33691.log 2024-03-16T10:12:40.679 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33143.log.gz 2024-03-16T10:12:40.679 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34296.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36100.log.gz 2024-03-16T10:12:40.679 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36627.log 2024-03-16T10:12:40.679 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34296.log.gz 2024-03-16T10:12:40.680 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.33691.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.32136.log 2024-03-16T10:12:40.680 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32641.log 2024-03-16T10:12:40.680 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33691.log.gz 2024-03-16T10:12:40.680 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32136.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36627.log.gz 2024-03-16T10:12:40.680 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32640.log 2024-03-16T10:12:40.680 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32136.log.gz 2024-03-16T10:12:40.680 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32841.log 2024-03-16T10:12:40.680 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34479.log 2024-03-16T10:12:40.680 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32640.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32641.log.gz 2024-03-16T10:12:40.681 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32841.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.32640.log.gz --verbose 2024-03-16T10:12:40.681 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.35347.log 2024-03-16T10:12:40.681 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32841.log.gz 2024-03-16T10:12:40.681 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34996.log 2024-03-16T10:12:40.681 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32545.log 2024-03-16T10:12:40.681 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35347.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34479.log.gz 2024-03-16T10:12:40.681 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36381.log 2024-03-16T10:12:40.681 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% 89.7% -- replaced with /var/log/ceph/ceph-client.admin.35347.log.gz 2024-03-16T10:12:40.681 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34996.log.gz 2024-03-16T10:12:40.682 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36256.log 2024-03-16T10:12:40.682 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36381.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-16T10:12:40.682 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32545.log.gz 2024-03-16T10:12:40.682 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36256.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36381.log.gz 2024-03-16T10:12:40.682 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35321.log 2024-03-16T10:12:40.682 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36256.log.gz 2024-03-16T10:12:40.682 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36575.log 2024-03-16T10:12:40.683 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34869.log 2024-03-16T10:12:40.683 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36575.log: gzip 88.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.35321.log.gz --verbose 2024-03-16T10:12:40.683 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.34510.log 2024-03-16T10:12:40.683 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36575.log.gz 2024-03-16T10:12:40.683 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32851.log 2024-03-16T10:12:40.683 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34869.log.gz 2024-03-16T10:12:40.683 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36861.log 2024-03-16T10:12:40.683 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35559.log 2024-03-16T10:12:40.684 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34510.log.gz 2024-03-16T10:12:40.684 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32851.log: /var/log/ceph/ceph-client.admin.36861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33169.log 2024-03-16T10:12:40.684 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36861.log.gz 2024-03-16T10:12:40.684 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32851.log.gz 2024-03-16T10:12:40.684 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35559.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-16T10:12:40.684 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33169.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35559.log.gz 2024-03-16T10:12:40.684 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- 89.0% /var/log/ceph/ceph-client.admin.35713.log -- replaced with /var/log/ceph/ceph-client.admin.33169.log.gz 2024-03-16T10:12:40.684 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-16T10:12:40.685 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33403.log 2024-03-16T10:12:40.685 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33273.log 2024-03-16T10:12:40.685 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35713.log.gz 2024-03-16T10:12:40.685 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34718.log 2024-03-16T10:12:40.685 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33403.log: /var/log/ceph/ceph-client.admin.33273.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.33403.log.gz 2024-03-16T10:12:40.685 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35403.log 2024-03-16T10:12:40.685 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33273.log.gz 2024-03-16T10:12:40.685 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34107.log 2024-03-16T10:12:40.686 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34718.log: /var/log/ceph/ceph-client.admin.35403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36074.log 2024-03-16T10:12:40.686 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34718.log.gz 2024-03-16T10:12:40.686 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35403.log.gz 2024-03-16T10:12:40.686 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35965.log 2024-03-16T10:12:40.686 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34107.log.gz 2024-03-16T10:12:40.686 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33299.log 2024-03-16T10:12:40.686 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35965.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36074.log.gz 2024-03-16T10:12:40.686 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35507.log 2024-03-16T10:12:40.686 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35965.log.gz 2024-03-16T10:12:40.687 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36290.log 2024-03-16T10:12:40.687 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33299.log: /var/log/ceph/ceph-client.admin.35507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34185.log 2024-03-16T10:12:40.687 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33299.log.gz 2024-03-16T10:12:40.687 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35507.log.gz 2024-03-16T10:12:40.687 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36809.log 2024-03-16T10:12:40.687 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36290.log: /var/log/ceph/ceph-client.admin.34185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36991.log 2024-03-16T10:12:40.687 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34185.log.gz 2024-03-16T10:12:40.687 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36290.log.gz 2024-03-16T10:12:40.687 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34640.log 2024-03-16T10:12:40.688 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36991.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36809.log.gz 2024-03-16T10:12:40.688 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32038.log 2024-03-16T10:12:40.688 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36991.log.gz 2024-03-16T10:12:40.688 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36913.log 2024-03-16T10:12:40.688 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34640.log: /var/log/ceph/ceph-client.admin.32038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34614.log 2024-03-16T10:12:40.688 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34640.log.gz 2024-03-16T10:12:40.688 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32038.log.gz 2024-03-16T10:12:40.688 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5/var/log/ceph/ceph-client.admin.36913.log: --verbose -- /var/log/ceph/ceph-client.admin.34744.log 2024-03-16T10:12:40.688 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35481.log 2024-03-16T10:12:40.689 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34614.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36913.log.gz 2024-03-16T10:12:40.689 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34159.log 2024-03-16T10:12:40.689 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34614.log.gz 2024-03-16T10:12:40.689 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34744.log.gz 2024-03-16T10:12:40.689 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33351.log 2024-03-16T10:12:40.689 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36407.log 2024-03-16T10:12:40.689 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34159.log: 89.0%/var/log/ceph/ceph-client.admin.33351.log: -- replaced with /var/log/ceph/ceph-client.admin.35481.log.gz 2024-03-16T10:12:40.689 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32647.log 2024-03-16T10:12:40.689 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34159.log.gz 2024-03-16T10:12:40.690 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33351.log.gz 2024-03-16T10:12:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35996.log 2024-03-16T10:12:40.690 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36407.log.gz 2024-03-16T10:12:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34666.log 2024-03-16T10:12:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32647.log: /var/log/ceph/ceph-client.admin.35996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33769.log 2024-03-16T10:12:40.690 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35996.log.gz 2024-03-16T10:12:40.690 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32647.log.gz 2024-03-16T10:12:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36653.log 2024-03-16T10:12:40.690 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34666.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-16T10:12:40.691 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33769.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34666.log.gz 2024-03-16T10:12:40.691 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35377.log 2024-03-16T10:12:40.691 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33769.log.gz 2024-03-16T10:12:40.691 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36653.log.gz 2024-03-16T10:12:40.691 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36835.log 2024-03-16T10:12:40.691 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34003.log 2024-03-16T10:12:40.691 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35377.log: /var/log/ceph/ceph-client.admin.36835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36230.log 2024-03-16T10:12:40.691 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35377.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36835.log.gz 2024-03-16T10:12:40.691 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34003.log: 2024-03-16T10:12:40.692 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33665.log 2024-03-16T10:12:40.692 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34003.log.gz 2024-03-16T10:12:40.692 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36022.log 2024-03-16T10:12:40.692 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36230.log: /var/log/ceph/ceph-client.admin.33665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34843.log 2024-03-16T10:12:40.692 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33665.log.gz 2024-03-16T10:12:40.692 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36230.log.gz 2024-03-16T10:12:40.692 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36152.log 2024-03-16T10:12:40.692 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34211.log 2024-03-16T10:12:40.692 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36022.log.gz 2024-03-16T10:12:40.693 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34843.log: /var/log/ceph/ceph-client.admin.36152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34453.log 2024-03-16T10:12:40.693 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34843.log.gz 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36152.log.gz 2024-03-16T10:12:40.693 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-16T10:12:40.693 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35585.log 2024-03-16T10:12:40.693 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34211.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22478.log 2024-03-16T10:12:40.693 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34453.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34211.log.gz 2024-03-16T10:12:40.693 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33615.log 2024-03-16T10:12:40.693 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34453.log.gz 2024-03-16T10:12:40.693 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35585.log.gz 2024-03-16T10:12:40.693 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph.tmp-client.admin.22478.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22478.log.gz 2024-03-16T10:12:40.694 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-16T10:12:40.694 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.0.33615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34588.log 2024-03-16T10:12:40.694 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34055.log 2024-03-16T10:12:40.694 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34588.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33484.log 2024-03-16T10:12:40.694 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.0.33615.log.gz 2024-03-16T10:12:40.694 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34588.log.gz 2024-03-16T10:12:40.694 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34055.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34055.log.gz 2024-03-16T10:12:40.702 INFO:teuthology.orchestra.run.smithi174.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-16T10:12:40.714 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35663.log 2024-03-16T10:12:40.714 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35126.log 2024-03-16T10:12:40.714 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4%/var/log/ceph/ceph-client.admin.35663.log: -- replaced with /var/log/ceph/ceph-client.admin.33484.log.gz 2024-03-16T10:12:40.714 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36783.log 2024-03-16T10:12:40.714 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35126.log: 91.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34375.log 2024-03-16T10:12:40.714 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35663.log.gz 2024-03-16T10:12:40.715 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35126.log.gz 2024-03-16T10:12:40.715 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36471.log 2024-03-16T10:12:40.715 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36783.log.gz 2024-03-16T10:12:40.715 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35861.log 2024-03-16T10:12:40.715 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5%/var/log/ceph/ceph-client.admin.36471.log: -- replaced with /var/log/ceph/ceph-client.admin.34375.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32519.log 2024-03-16T10:12:40.715 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-16T10:12:40.715 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36471.log.gz 2024-03-16T10:12:40.715 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35861.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35861.log.gz 2024-03-16T10:12:40.716 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32571.log 2024-03-16T10:12:40.716 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34427.log 2024-03-16T10:12:40.716 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32519.log: /var/log/ceph/ceph-client.admin.32571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33429.log 2024-03-16T10:12:40.716 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34427.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32519.log.gz 2024-03-16T10:12:40.716 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32571.log.gz 2024-03-16T10:12:40.716 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 87.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.34427.log.gz /var/log/ceph/ceph.log 2024-03-16T10:12:40.716 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-16T10:12:40.716 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33977.log 2024-03-16T10:12:40.716 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33429.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37017.log 2024-03-16T10:12:40.717 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33977.log: 84.6% 90.9% -- replaced with /var/log/ceph/ceph-client.admin.33429.log.gz 2024-03-16T10:12:40.717 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33977.log.gz 2024-03-16T10:12:40.717 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-03-16T10:12:40.717 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34244.log 2024-03-16T10:12:40.717 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32644.log 2024-03-16T10:12:40.717 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37017.log: /var/log/ceph/ceph-client.admin.34244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36965.log 2024-03-16T10:12:40.717 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32644.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34244.log.gz 2024-03-16T10:12:40.717 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37017.log.gz 2024-03-16T10:12:40.718 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32644.log.gz 2024-03-16T10:12:40.718 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36126.log 2024-03-16T10:12:40.718 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36204.log 2024-03-16T10:12:40.718 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36965.log: /var/log/ceph/ceph-client.admin.36126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36327.log 2024-03-16T10:12:40.718 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36204.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36965.log.gz 2024-03-16T10:12:40.718 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36126.log.gz 2024-03-16T10:12:40.718 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33247.log 88.7% 2024-03-16T10:12:40.718 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36204.log.gz 2024-03-16T10:12:40.718 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32162.log 2024-03-16T10:12:40.719 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36327.log: /var/log/ceph/ceph-client.admin.33247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35769.log 2024-03-16T10:12:40.719 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32162.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33247.log.gz 2024-03-16T10:12:40.719 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36327.log.gz 2024-03-16T10:12:40.719 INFO:teuthology.orchestra.run.smithi017.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35074.log 2024-03-16T10:12:40.719 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32162.log.gz 2024-03-16T10:12:40.719 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32648.log 2024-03-16T10:12:40.719 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.0.35769.log: /var/log/ceph/ceph-client.admin.35074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35260.log 2024-03-16T10:12:40.719 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35074.log.gz/var/log/ceph/ceph-client.admin.32648.log: 2024-03-16T10:12:40.719 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32648.log.gz 2024-03-16T10:12:40.720 INFO:teuthology.orchestra.run.smithi017.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.35769.log.gz 2024-03-16T10:12:40.720 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32188.log 2024-03-16T10:12:40.720 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35455.log 2024-03-16T10:12:40.720 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32188.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35260.log.gz 2024-03-16T10:12:40.720 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32858.log 2024-03-16T10:12:40.720 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33899.log 2024-03-16T10:12:40.720 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32188.log.gz 2024-03-16T10:12:40.720 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35455.log: /var/log/ceph/ceph-client.admin.32858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36731.log 2024-03-16T10:12:40.720 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32858.log.gz 2024-03-16T10:12:40.721 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33899.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35455.log.gz 2024-03-16T10:12:40.721 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32357.log 2024-03-16T10:12:40.721 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33899.log.gz 2024-03-16T10:12:40.721 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36048.log 2024-03-16T10:12:40.721 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36731.log: /var/log/ceph/ceph-client.admin.32357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33951.log 2024-03-16T10:12:40.721 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36048.log: 89.0% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36731.log.gz 2024-03-16T10:12:40.721 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32357.log.gz 2024-03-16T10:12:40.721 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36048.log.gz 2024-03-16T10:12:40.721 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32415.log 2024-03-16T10:12:40.722 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32645.log 2024-03-16T10:12:40.722 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33951.log: /var/log/ceph/ceph-client.admin.32415.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33951.log.gz 2024-03-16T10:12:40.722 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32415.log.gz 2024-03-16T10:12:40.722 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36355.log 2024-03-16T10:12:40.722 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33925.log 2024-03-16T10:12:40.722 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32645.log: /var/log/ceph/ceph-client.admin.36355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34944.log 2024-03-16T10:12:40.722 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33925.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32645.log.gz 2024-03-16T10:12:40.722 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36355.log.gz 2024-03-16T10:12:40.722 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35048.log 2024-03-16T10:12:40.723 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33925.log.gz 2024-03-16T10:12:40.723 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27355.log 2024-03-16T10:12:40.723 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34944.log: /var/log/ceph/ceph-client.admin.35048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33821.log 2024-03-16T10:12:40.723 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5%/var/log/ceph/ceph-client.admin.27355.log: -- replaced with /var/log/ceph/ceph-client.admin.35048.log.gz 2024-03-16T10:12:40.723 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34944.log.gz 2024-03-16T10:12:40.723 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27355.log.gz 2024-03-16T10:12:40.723 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32110.log 2024-03-16T10:12:40.723 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32084.log 2024-03-16T10:12:40.723 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33821.log: /var/log/ceph/ceph-client.admin.32110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34401.log 2024-03-16T10:12:40.724 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32084.log: 88.9% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33821.log.gz 2024-03-16T10:12:40.724 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32110.log.gz 2024-03-16T10:12:40.724 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32084.log.gz 2024-03-16T10:12:40.724 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35533.log 2024-03-16T10:12:40.724 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34081.log 2024-03-16T10:12:40.724 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34401.log: /var/log/ceph/ceph-client.admin.35533.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-16T10:12:40.724 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34081.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35533.log.gz 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34401.log.gz 2024-03-16T10:12:40.724 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-16T10:12:40.724 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34081.log.gz 2024-03-16T10:12:40.724 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33795.log 2024-03-16T10:12:40.725 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32007.log 2024-03-16T10:12:40.725 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.33795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34970.log 2024-03-16T10:12:40.725 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32007.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33795.log.gz 2024-03-16T10:12:40.725 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32007.log.gz 2024-03-16T10:12:40.725 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-16T10:12:40.725 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35295.log 2024-03-16T10:12:40.725 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-16T10:12:40.725 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34970.log: /var/log/ceph/ceph-client.admin.35295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33117.log 2024-03-16T10:12:40.725 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mds.a.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35295.log.gz 2024-03-16T10:12:40.725 INFO:teuthology.orchestra.run.smithi017.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34970.log.gz 2024-03-16T10:12:40.726 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32597.log 2024-03-16T10:12:40.726 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36757.log 2024-03-16T10:12:40.726 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33117.log: /var/log/ceph/ceph-client.admin.32597.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32597.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33117.log.gz 2024-03-16T10:12:40.726 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-16T10:12:40.726 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33717.log 2024-03-16T10:12:40.726 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33221.log 2024-03-16T10:12:40.726 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36757.log: /var/log/ceph/ceph-client.admin.33717.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36757.log.gz 2024-03-16T10:12:40.726 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33717.log.gz 2024-03-16T10:12:40.726 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33039.log 2024-03-16T10:12:40.727 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34898.log 2024-03-16T10:12:40.727 INFO:teuthology.orchestra.run.smithi174.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-16T10:12:40.741 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33221.log: /var/log/ceph/ceph-client.admin.33039.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33221.log.gz 2024-03-16T10:12:40.741 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33039.log.gz 2024-03-16T10:12:40.741 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33873.log 2024-03-16T10:12:40.741 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32862.log 2024-03-16T10:12:40.741 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33873.log: /var/log/ceph/ceph-client.admin.34898.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34898.log.gz 2024-03-16T10:12:40.742 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33873.log.gz 2024-03-16T10:12:40.742 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36523.log 2024-03-16T10:12:40.742 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31951.log 2024-03-16T10:12:40.742 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32862.log: /var/log/ceph/ceph-client.admin.36523.log: 89.4% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32862.log.gz 2024-03-16T10:12:40.742 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36523.log.gz 2024-03-16T10:12:40.742 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35887.log 2024-03-16T10:12:40.742 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35178.log 2024-03-16T10:12:40.742 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.31951.log: /var/log/ceph/ceph-client.admin.35887.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31951.log.gz 2024-03-16T10:12:40.743 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35887.log.gz 2024-03-16T10:12:40.743 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27465.log 2024-03-16T10:12:40.743 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32331.log 2024-03-16T10:12:40.743 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35178.log: /var/log/ceph/ceph-client.admin.27465.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35178.log.gz 2024-03-16T10:12:40.743 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27465.log.gz 2024-03-16T10:12:40.743 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34692.log 2024-03-16T10:12:40.743 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36601.log 2024-03-16T10:12:40.743 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32331.log: /var/log/ceph/ceph-client.admin.34692.log: 88.3% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32331.log.gz 2024-03-16T10:12:40.743 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34692.log.gz 2024-03-16T10:12:40.743 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32441.log 2024-03-16T10:12:40.744 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36445.log 2024-03-16T10:12:40.744 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36601.log: /var/log/ceph/ceph-client.admin.32441.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36601.log.gz 2024-03-16T10:12:40.744 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32441.log.gz 2024-03-16T10:12:40.744 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32383.log 2024-03-16T10:12:40.744 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37043.log 2024-03-16T10:12:40.744 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36445.log: /var/log/ceph/ceph-client.admin.32383.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32383.log.gz 88.9% 2024-03-16T10:12:40.744 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36445.log.gz 2024-03-16T10:12:40.744 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33195.log 2024-03-16T10:12:40.744 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35611.log 2024-03-16T10:12:40.745 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.37043.log: /var/log/ceph/ceph-client.admin.33195.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37043.log.gz 2024-03-16T10:12:40.745 INFO:teuthology.orchestra.run.smithi017.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33195.log.gz 2024-03-16T10:12:40.745 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33325.log 2024-03-16T10:12:40.745 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32651.log 2024-03-16T10:12:40.745 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35611.log: /var/log/ceph/ceph-client.admin.33325.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35611.log.gz 2024-03-16T10:12:40.745 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33325.log.gz 2024-03-16T10:12:40.745 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32834.log 2024-03-16T10:12:40.745 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32651.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32651.log.gz 2024-03-16T10:12:40.745 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32296.log 2024-03-16T10:12:40.746 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36939.log 2024-03-16T10:12:40.746 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32834.log: /var/log/ceph/ceph-client.admin.32296.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32834.log.gz 2024-03-16T10:12:40.746 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32296.log.gz 2024-03-16T10:12:40.746 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32832.log 2024-03-16T10:12:40.746 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36178.log 2024-03-16T10:12:40.746 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36939.log: /var/log/ceph/ceph-client.admin.32832.log: 89.4% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36939.log.gz 2024-03-16T10:12:40.746 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32832.log.gz 2024-03-16T10:12:40.746 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34536.log 2024-03-16T10:12:40.747 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32268.log 2024-03-16T10:12:40.747 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36178.log: /var/log/ceph/ceph-client.admin.34536.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36178.log.gz 2024-03-16T10:12:40.747 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.34536.log.gz 2024-03-16T10:12:40.747 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36887.log 2024-03-16T10:12:40.747 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32268.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32268.log.gz 2024-03-16T10:12:40.747 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32493.log 2024-03-16T10:12:40.747 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32635.log 2024-03-16T10:12:40.747 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36887.log: /var/log/ceph/ceph-client.admin.32493.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36887.log.gz 2024-03-16T10:12:40.747 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32493.log.gz 2024-03-16T10:12:40.748 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32214.log 2024-03-16T10:12:40.748 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33065.log 2024-03-16T10:12:40.748 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32635.log: /var/log/ceph/ceph-client.admin.32214.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32635.log.gz 2024-03-16T10:12:40.748 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32214.log.gz 2024-03-16T10:12:40.748 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34322.log 2024-03-16T10:12:40.748 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34562.log 2024-03-16T10:12:40.748 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33065.log: /var/log/ceph/ceph-client.admin.34322.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33065.log.gz 2024-03-16T10:12:40.748 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34322.log.gz 2024-03-16T10:12:40.748 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33847.log 2024-03-16T10:12:40.749 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33377.log 2024-03-16T10:12:40.749 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34562.log: /var/log/ceph/ceph-client.admin.33847.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34562.log.gz 2024-03-16T10:12:40.749 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33847.log.gz 2024-03-16T10:12:40.749 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35232.log 2024-03-16T10:12:40.749 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34029.log 2024-03-16T10:12:40.749 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33377.log: /var/log/ceph/ceph-client.admin.35232.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33377.log.gz 2024-03-16T10:12:40.749 INFO:teuthology.orchestra.run.smithi017.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35232.log.gz 2024-03-16T10:12:40.749 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33743.log 2024-03-16T10:12:40.750 INFO:teuthology.orchestra.run.smithi174.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-16T10:12:40.750 INFO:teuthology.orchestra.run.smithi174.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-16T10:12:40.751 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35831.log 2024-03-16T10:12:40.751 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34029.log: /var/log/ceph/ceph-client.admin.33743.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34029.log.gz 2024-03-16T10:12:40.752 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33743.log.gz 2024-03-16T10:12:40.752 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35831.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35831.log.gz 2024-03-16T10:12:40.752 INFO:teuthology.orchestra.run.smithi017.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-16T10:12:40.752 INFO:teuthology.orchestra.run.smithi017.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-16T10:12:40.757 INFO:teuthology.orchestra.run.smithi174.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-16T10:12:40.762 INFO:teuthology.orchestra.run.smithi174.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-16T10:12:40.764 INFO:teuthology.orchestra.run.smithi017.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-16T10:12:40.783 INFO:teuthology.orchestra.run.smithi017.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-16T10:12:40.808 INFO:teuthology.orchestra.run.smithi174.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-16T10:12:40.810 INFO:teuthology.orchestra.run.smithi174.stderr: 2024-03-16T10:12:40.810 INFO:teuthology.orchestra.run.smithi174.stderr:real 0m0.183s 2024-03-16T10:12:40.810 INFO:teuthology.orchestra.run.smithi174.stderr:user 0m0.684s 2024-03-16T10:12:40.810 INFO:teuthology.orchestra.run.smithi174.stderr:sys 0m0.048s 2024-03-16T10:12:40.813 INFO:teuthology.orchestra.run.smithi017.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-16T10:12:40.856 INFO:teuthology.orchestra.run.smithi017.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-16T10:12:40.858 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-03-16T10:12:40.858 INFO:teuthology.orchestra.run.smithi017.stderr:real 0m0.238s 2024-03-16T10:12:40.858 INFO:teuthology.orchestra.run.smithi017.stderr:user 0m0.872s 2024-03-16T10:12:40.858 INFO:teuthology.orchestra.run.smithi017.stderr:sys 0m0.112s 2024-03-16T10:12:40.858 INFO:tasks.ceph:Archiving logs... 2024-03-16T10:12:40.858 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360/remote/smithi017/log 2024-03-16T10:12:40.859 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T10:12:41.115 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360/remote/smithi174/log 2024-03-16T10:12:41.116 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T10:12:41.203 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-16T10:12:41.222 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-16T10:12:41.223 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T10:12:41.225 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T10:12:41.299 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-16T10:12:41.307 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-16T10:12:41.307 DEBUG:teuthology.orchestra.run.smithi017:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T10:12:41.310 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T10:12:41.320 INFO:teuthology.orchestra.run.smithi017.stdout: remote refid st t when poll reach delay offset jitter 2024-03-16T10:12:41.320 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================== 2024-03-16T10:12:41.320 INFO:teuthology.orchestra.run.smithi017.stdout:*hv01.front.sepi 44.4.53.2 2 u 51 64 37 0.101 -1.497 1.694 2024-03-16T10:12:41.320 INFO:teuthology.orchestra.run.smithi017.stdout:+hv02.front.sepi 63.231.80.2 3 u 50 64 37 0.113 +1.434 1.622 2024-03-16T10:12:41.320 INFO:teuthology.orchestra.run.smithi017.stdout:+hv03.front.sepi 74.6.168.72 3 u 43 64 37 0.090 +0.319 1.649 2024-03-16T10:12:41.321 INFO:teuthology.orchestra.run.smithi017.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T10:12:41.322 INFO:teuthology.orchestra.run.smithi174.stdout: remote refid st t when poll reach delay offset jitter 2024-03-16T10:12:41.323 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================== 2024-03-16T10:12:41.323 INFO:teuthology.orchestra.run.smithi174.stdout:*hv01.front.sepi 44.4.53.2 2 u 51 64 37 0.140 -2.201 0.270 2024-03-16T10:12:41.323 INFO:teuthology.orchestra.run.smithi174.stdout:+hv02.front.sepi 63.231.80.2 3 u 50 64 37 0.137 +0.750 0.314 2024-03-16T10:12:41.323 INFO:teuthology.orchestra.run.smithi174.stdout:+hv03.front.sepi 74.6.168.72 3 u 52 64 37 0.107 -0.358 0.289 2024-03-16T10:12:41.323 INFO:teuthology.orchestra.run.smithi174.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T10:12:41.324 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-16T10:12:41.335 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-16T10:12:41.335 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-16T10:12:41.344 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-16T10:12:41.353 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-16T10:12:41.362 INFO:teuthology.task.internal:Duration was 744.528020 seconds 2024-03-16T10:12:41.362 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-16T10:12:41.371 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-16T10:12:41.371 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T10:12:41.374 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T10:12:41.411 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-16T10:12:41.412 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi017.front.sepia.ceph.com 2024-03-16T10:12:41.412 DEBUG:teuthology.orchestra.run.smithi017:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T10:12:41.465 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2024-03-16T10:12:41.465 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T10:12:41.475 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-16T10:12:41.476 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T10:12:41.513 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T10:12:41.544 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-16T10:12:41.544 DEBUG:teuthology.orchestra.run.smithi017:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T10:12:41.581 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T10:12:41.642 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-16T10:12:41.652 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-16T10:12:41.652 DEBUG:teuthology.orchestra.run.smithi017:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T10:12:41.664 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T10:12:41.677 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-16T10:12:41.687 DEBUG:teuthology.orchestra.run.smithi017:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T10:12:41.709 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T10:12:41.717 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = core 2024-03-16T10:12:41.729 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2024-03-16T10:12:41.751 DEBUG:teuthology.orchestra.run.smithi017:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T10:12:41.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:12:41.775 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T10:12:41.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T10:12:41.803 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-16T10:12:41.812 INFO:teuthology.task.internal:Transferring archived files... 2024-03-16T10:12:41.813 DEBUG:teuthology.misc:Transferring archived files from smithi017:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360/remote/smithi017 2024-03-16T10:12:41.813 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T10:12:41.840 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360/remote/smithi174 2024-03-16T10:12:41.840 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T10:12:41.867 INFO:teuthology.task.internal:Removing archive directory... 2024-03-16T10:12:41.867 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T10:12:41.885 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T10:12:41.911 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-16T10:12:41.922 INFO:teuthology.task.internal:Not uploading archives. 2024-03-16T10:12:41.922 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-16T10:12:41.933 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-16T10:12:41.933 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T10:12:41.935 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T10:12:41.940 INFO:teuthology.orchestra.run.smithi017.stdout: 658465 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 10:12 /home/ubuntu/cephtest 2024-03-16T10:12:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: 658634 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 10:12 /home/ubuntu/cephtest 2024-03-16T10:12:41.961 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-16T10:12:41.971 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-16T10:12:41.990 INFO:teuthology.nuke:Checking targets against current locks 2024-03-16T10:12:42.018 DEBUG:teuthology.nuke:shortname: smithi017 2024-03-16T10:12:42.018 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T10:12:42.043 DEBUG:teuthology.nuke:shortname: smithi174 2024-03-16T10:12:42.044 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T10:12:42.047 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 09:51:15.341845', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAzG3WuJoO9CsYpabwEtXZIEsNWtuwVD9c72O5USbYQ8CXtNNbr/289XNGIWmEdcOxZjFtJOiM+NbxM2k1OpnIk='} 2024-03-16T10:12:42.068 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-15_13:00:41-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7602360', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 09:51:15.342971', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKvDj+uaglxBy4cNsSC8W66C4LA0RrviET8P2Pr+TEqKog/O1npB2zywdCmM4QOM8TIu5s5VxLpqZoxG/2ZLGzg='} 2024-03-16T10:12:42.101 INFO:teuthology.orchestra.console.smithi017:Power off 2024-03-16T10:12:42.101 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T10:12:42.123 DEBUG:teuthology.orchestra.console.smithi017:power off output: Chassis Power Control: Down/Off 2024-03-16T10:12:42.123 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:12:42.137 INFO:teuthology.orchestra.console.smithi174:Power off 2024-03-16T10:12:42.138 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T10:12:42.148 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-03-16T10:12:42.160 DEBUG:teuthology.orchestra.console.smithi174:power off output: Chassis Power Control: Down/Off 2024-03-16T10:12:42.161 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:12:42.183 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-03-16T10:12:46.148 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:12:46.185 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:12:46.259 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is on 2024-03-16T10:12:46.294 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-03-16T10:12:50.260 DEBUG:teuthology.orchestra.console.smithi017:pexpect command: ipmitool -H smithi017.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:12:50.294 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:12:50.370 DEBUG:teuthology.orchestra.console.smithi017:check power output: Chassis Power is off 2024-03-16T10:12:50.403 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is on 2024-03-16T10:12:50.472 INFO:teuthology.orchestra.console.smithi017:Power off completed 2024-03-16T10:12:54.408 DEBUG:teuthology.orchestra.console.smithi174:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T10:12:54.518 DEBUG:teuthology.orchestra.console.smithi174:check power output: Chassis Power is off 2024-03-16T10:12:54.619 INFO:teuthology.orchestra.console.smithi174:Power off completed 2024-03-16T10:12:54.723 INFO:teuthology.run:Summary data: description: fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} duration: 744.5280199050903 failure_reason: 'Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD)' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=f2e18ce7b61f40ef91ba7c653e2538a2 status: fail success: false 2024-03-16T10:12:54.724 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T10:12:54.794 INFO:teuthology.run:FAIL